Benchmarking
- Akash Kumar Jaiswal (Unlicensed)
- Ashhad Alam
Adeptia conducts extensive performance testing with each release to ensure improving and better performance of its product. Some performance benchmarks are provided here for reference:
Test A - Large Number of Small EDI Files
System Configuration
Cluster = 2 nodes (Kernel + WebRunner)
Processor = Intel Xeon CPU E5-2686 2.30GHzÂ
Number Of Cores = 4
Total RAM = 16GB
OS = Windows Server 2012 R2 64-bit
Backend / Log Database - AWS SQL RDS
Kernel = 8GB
Result -Â All tests ran successfully
Â
Scenario | File Size x # of Files | Total Time | Throughput |
|---|---|---|---|
Inbound File Type – EDI 4010 Orders Mapping – Medium Complex # of ST = 1 | 2KB x 3000 | 13m | 4 files/sec |
Inbound File Type – EDI 4010 Orders Mapping – Medium Complex # of ST = 1 | 2KB x 6000 | 26m | 4 files/sec  |
Inbound File Type – EDI 4010 Orders Mapping – Medium Complex # of ST = 1 | 2KB x 20,000 | 1h 46m | 3.14 files/sec  |
Inbound File Type – EDI 4010 Orders Mapping – Medium Complex # of ST = 44 | 37KB x 1000 | 1h 12m | 4 files/sec  |
Outbound EDI from IDOC File Type – EDI 4010 Invoice Mapping – Medium Complex | 11KB x 3000 | 12m 08s | 4.12 files/sec  |
Test B -Â Large Sized EDI Files
System Configuration
- Single node (no cluster)
- Processor = Intel Xeon CPU E5-2686 2.30GHzÂ
- Number Of Cores = 4
- Total RAM = 16GB
- OS = Windows Server 2012 R2 64-bit
- Backend / Log Database - AWS Oracle RDS
- Kernel = 8GB
Result: All tests ran successfully
Â
Scenario | File Size x # of Files | Max CPU % | Max RAM % | Max. Kernel Heap | Total Time |
|---|---|---|---|---|---|
Inbound File Type – EDI 4010 ISA : 1 GS : 1 Transaction Set : 5000 ~ 25-35 segments in each transaction set  | 2.79MB x 1 | 42% | 38% | 1.1GB | 34s |
Inbound File Type – EDI 4010 ISA : 1 GS : 13 Transaction Set : 1 ST in each GS ~ 140000 segments in each transaction set  | 40.1MB x 1 | 88% | 46% | 2.3GB | 5m 22s |
Inbound File Type – EDI 4010 ISA : 1 GS : 1 Transaction Set : 10000 ~ 50-103 segments in each Transaction set  | 13.6MB x 1 | 72% | 43% | 2.2GB | 1m 31s |
Test C - Large File Processing with Large Data Ingestion
System Configuration
- Single node (no cluster)
- Processor = Intel Xeon CPU E5-2686 2.30GHzÂ
- Number Of Cores = 4
- Total RAM = 16GB
- OS = Windows Server 2012 R2 64-bit
- Backend / Log Database - AWS Oracle RDS
- Kernel = 8GB
Result: All test ran successfully
Â
Scenario | File Size x # of Files | Number of Records per file | Max CPU % | Max RAM % | Time |
|---|---|---|---|---|---|
File Type – XML Mapping – Simple 1-to-1 # of Files = 1  | 25GB x 1 | 696895 | 27% | 34% | 23m 48s |
50GB x 1 | 1393690 | 27% | 34% | 47m 30s | |
200GB x 1 | 5574760 | 27% | 34% | 3h 10m | |
500GB x 1 | 13936900 | 27% | 34% | 7h 52m | |
File Type – XML Mapping – Simple 1-to-1 # of Files = 50 parallel | 25GB x 50 | 696895 | 100% | 37% | 10h 7m (total) 8h 51m (avg/file) |
50GB x 50 | 1393690 | 100% | 37% | 19h 45m (total) 17h 13m (avg/file) | |
File Type – XML Mapping – Complex (Functions used: Math, String, DB Look, If Condition) # of Files = 1 | 25GB x 1 | 696895 | 29% | 36% | 33m 30s |
50GB x 1 | 1393690 | 29% | 36% | 1h 7m | |
200GB x 1 | 5574760 | 29% | 36% | 4h 27m | |
500GB x 1 | 13936900 | 29% | 36% | 11h 13m | |
File Type – XML Mapping – Complex # of Files = 50 parallel | 25GB x 50 | 696895 | 100% | 39% | 9h 59m (total) 8h 22m (avg/file) |
50GB x 50 | 1393690 | 100% | 39% | 20h 2m (total) 15h 53m (avg/file) | |
File Type – TEXT Mapping – Simple 1-to-1 # of Files = 1 | 25GB x 1 | 696895 | 45% | 36% | 6h 26m |
50GB x 1 | 1393690 | 45% | 36% | 12h 40m | |
200GB x 1 | 5574760 | 45% | 36% | 51h 30m |
Test D - UI Load Testing with Concurrent Users
System Configuration
- Single node (no cluster)
- Processor = Intel Xeon CPU E5-2686 2.30GHzÂ
- Number Of Cores = 4
- Total RAM = 16GB
- OS = Linux 64-bit
- Backend / Log Database - AWS Oracle RDS
- Kernel = 8GB
- WebRunner = 4GB
- Tool used = JMeter
Result: All tests ran successfully
Â
Scenario | Home Page | Dashboard | Network | Partners | Transactions | Max CPU % | Max RAM % |
|---|---|---|---|---|---|---|---|
# of Partners and Transactions in Back-end DB = 1138 # of Concurrent Users = 25 # of Loops (iterations) = 50 Â | 4.2s | 1.1s | 0.8s | 0.4s | 0.6s | 53% | 64% |
# of Partners and Transactions in Back-end DB = 1638 # of Concurrent Users = 25 # of Loops (iterations) = 50 Â | 4.3s | 1.0s | 0.9s | 0.3s | 0.4s | 49% | 59% |
# of Partners and Transactions in Back-end DB = 2200 # of Concurrent Users = 25 # of Loops (iterations) = 50 Â | 2.3s | Overview page : 4.3s | 9.0s | 0.7s | 2.7s | 64% | 59% |
| Transaction page: 0.4s | |||||||
| Trigger page: 0.5s |
Test E - Advanced Positional File
System Configuration
Single node (no cluster)
Processor = Intel Xeon CPU E5-2686 2.30GHzÂ
Number Of Cores = 4Â
Total RAM =64GBÂ
OS = Windows Server 2012 R2 64-bit
Backend / Log Database - AWS Oracle RDS
Kernel = 20GB (minimum), 40GB (maximum)
Result -Â All tests ran successfullyÂ
Scenario | Number of Records per file | File Size x # of Files | Total Time |
|---|---|---|---|
File Type – Text to Advanced Positional Mapping – Simple 1-to-1 | 3 million | 234 Mb x 1 | 45m 39s |
File Type – Text to Advanced Positional Mapping – Simple 1-to-1 | 5 million | 391 Mb x 1 | 1h 19m 31s |
File Type – Text to Advanced Positional Mapping – Simple 1-to-1 | 10 million | 782 Mb x 1 | 2h 31m 49s |
File Type – Text to Advanced Positional Mapping – Simple 1-to-1 | 10 million | 782 Mb x 3 | 2h 13m 4s |
File Type – Text to Advanced Positional Mapping – Simple 1-to-1 | 10 million | 782 Mb x 5 | 2h 12m 37s |
File Type – Text to Advanced Positional Mapping – Simple 1-to-1 | 10 million | 782 Mb x 10 | 2h 42m 45s |
File Type – Text to Advanced Positional Mapping – Simple 1-to-1 | 10 million | 782 Mb x 15 | 3h 56m 36s |
Test F - API Load Testing Results
Click here to open the test results page.Â
Test G - Schema
| Sl. No. | Test Case Description | Data Set (File Size) | Execution Time |
|---|---|---|---|
| 1      | To test processing of simple Excel file (xls) through File Event, File Source and File target Process Flow: Performance_Process_flow_Excel | 100 MB      | 1 min 28 sec Kernel CPU: 25% Kernel Mem: 5.5GB |
|  2 | To test processing of simple Positional file through File Event, File Source and File target Process Flow: Performance_Process_flow_Positional |  100 MB | 59 sec Kernel CPU: 27% Kernel Mem: 7.21GB |
|  3 | To test processing of simple JSON file through File Event, File Source and File target Process Flow: Performance_Process_flow_JSON |  100 MB | 25 sec Kernel CPU: 3% Kernel Mem: 8.7GB |
|  4 | To test processing of simple Adv Positional file through File Event, File Source and File target Process Flow: Performance_Process_flow_Adv_Positional |  100 MB | 1 min 15 sec Kernel CPU: 29% Kernel Mem: 8.2GB |
| 5 | To test processing of simple Text file through File Event, File Source and File target Process Flow: Performance_Process_flow_Text |  100 MB | 2 min 00 sec Kernel CPU: 28.5% Kernel Mem: 8.5GB |
| 6 | To test processing of simple Adv. Text file through File Event, File Source and File target Process Flow: Performance_Process_flow_adv_text | 100 MB | 1 min 49 sec Kernel CPU: 25% Kernel Mem: 8.6GB |
| 7 | To test processing of simple xml file through File Event, File Source and File target Process Flow: Performance_Process_flow_Xml | 100 MB | 7 sec Kernel CPU: 19% Kernel Mem: 8.85GB |
| 8 | To test Database Event along with Adv Database Source when the number of records to pick up are large in the table. Use one to map to write data to Adv DB target with different batch/commit size options. Process Flow: Performance_Process_flow_Adv_Database | 50000 records to pickup Batch size=5000 Commit Size=2500 | 1 min 36 sec Kernel CPU: 19.8% Kernel Mem: 3.56GB |
| 9 | To test Database Event along with Adv Database Source when the number of records to pick up are large in the table. Use one to map to write data to Adv DB target with different batch/commit size options. Process Flow: Performance_Process_flow_Adv_Database | 50000 records to pickup Batch size=5000 Commit Size=5000 | 1 min 40 sec Kernel CPU: 20.2% Kernel Mem: 3.34GB |
Test H - Miscellaneous
| Sl. No. | Test Case Description | Data Set (File Size) | Execution Time |
|---|---|---|---|
| 1 | To test concurrent processing of flat files of different sizes with moderate mapping functions and one to one conversion. | File Type: CSV File Size: 1MB Concurrent Files: 500 | 2 min 54 sec Kernel CPU: 5.20% Kernel Mem: 3.33GB |
| 2 | To test concurrent processing of flat files of different sizes with moderate mapping functions and one to one conversion. | File Type: CSV File Size: 1MB Concurrent Files: 1000 | 9 min 32 sec Kernel CPU: 5.82% Kernel Mem: 3.34GB |
| 3 | To test concurrent processing of flat files of different sizes with moderate mapping functions and one to one conversion. | File Type: CSV File Size: 100KB Concurrent Files: 5000 | |
| 4 | To test processing of flat files located at File through non-streaming Mapping with pre-built Mapping functions applied (string, conditional, date, math) and generate the Target file. | 100MB Plain CSV Text file Non-streaming Mapping mode | 1 min 40 sec Kernel CPU: 2.5% Kernel Mem: 3.37GB |