Benchmarking
- Akash Kumar Jaiswal (Unlicensed)
Adeptia conducts extensive performance testing with each release to ensure improving and better performance of its product. Some performance benchmarks are provided here for reference:
Test A - Large Number of Small EDI Files
System Configuration
Cluster = 2 nodes (Kernel + WebRunner)
Processor = Intel Xeon CPU E5-2686 2.30GHz
Number Of Cores = 4
Total RAM = 16GB
OS = Windows Server 2012 R2 64-bit
Backend / Log Database - AWS SQL RDS
Kernel = 8GB
Result - All tests ran successfully
Scenario | File Size x # of Files | Total Time | Throughput |
|---|---|---|---|
Inbound File Type – EDI 4010 Orders Mapping – Medium Complex # of ST = 1 | 2KB x 3000 | 13m | 4 files/sec |
Inbound File Type – EDI 4010 Orders Mapping – Medium Complex # of ST = 1 | 2KB x 6000 | 26m | 4 files/sec
|
Inbound File Type – EDI 4010 Orders Mapping – Medium Complex # of ST = 1 | 2KB x 20,000 | 1h 45m | 3 files/sec
|
Inbound File Type – EDI 4010 Orders Mapping – Medium Complex # of ST = 44 | 37KB x 1000 | 1h 12m | 4 files/sec
|
Outbound EDI from IDOC File Type – EDI 4010 Invoice Mapping – Medium Complex | 14KB x 3000 | 9m 2s | 6 files/sec
|
Test B - Large Sized EDI Files
System Configuration
- Single node (no cluster)
- Processor = Intel Xeon CPU E5-2686 2.30GHz
- Number Of Cores = 4
- Total RAM = 16GB
- OS = Windows Server 2012 R2 64-bit
- Backend / Log Database - AWS Oracle RDS
- Kernel = 8GB
Result: All tests ran successfully
Scenario | File Size x # of Files | Max CPU % | Max RAM % | Max. Kernel Heap | Total Time |
|---|---|---|---|---|---|
Inbound File Type – EDI 4010 ISA : 1 GS : 1 Transaction Set : 5000 ~ 25-35 segments in each transaction set
| 2.79MB x 1 | 42% | 38% | 1.1GB | 34s |
Inbound File Type – EDI 4010 ISA : 1 GS : 13 Transaction Set : 1 ST in each GS ~ 140000 segments in each transaction set
| 40.1MB x 1 | 88% | 46% | 2.3GB | 3m |
Inbound File Type – EDI 4010 ISA : 1 GS : 1 Transaction Set : 10000 ~ 50-103 segments in each Transaction set
| 13.6MB x 1 | 72% | 43% | 2.2GB | 1m 30s |
Test C - Large File Processing with Large Data Ingestion
System Configuration
- Single node (no cluster)
- Processor = Intel Xeon CPU E5-2686 2.30GHz
- Number Of Cores = 4
- Total RAM = 16GB
- OS = Windows Server 2012 R2 64-bit
- Backend / Log Database - AWS Oracle RDS
- Kernel = 8GB
Result: All test ran successfully
Scenario | File Size x # of Files | Number of Records per file | Max CPU % | Max RAM % | Time |
|---|---|---|---|---|---|
File Type – XML Mapping – Simple 1-to-1 # of Files = 1
| 25GB x 1 | 696895 | 27% | 34% | 23m 48s |
50GB x 1 | 1393690 | 27% | 34% | 47m 30s | |
200GB x 1 | 5574760 | 27% | 34% | 3h 10m | |
500GB x 1 | 13936900 | 27% | 34% | 7h 52m | |
File Type – XML Mapping – Simple 1-to-1 # of Files = 50 parallel | 25GB x 50 | 696895 | 100% | 37% | 10h 7m (total) 8h 51m (avg/file) |
50GB x 50 | 1393690 | 100% | 37% | 19h 45m (total) 17h 13m (avg/file) | |
File Type – XML Mapping – Complex (Functions used: Math, String, DB Look, If Condition) # of Files = 1 | 25GB x 1 | 696895 | 29% | 36% | 33m 30s |
50GB x 1 | 1393690 | 29% | 36% | 1h 7m | |
200GB x 1 | 5574760 | 29% | 36% | 4h 27m | |
500GB x 1 | 13936900 | 29% | 36% | 11h 13m | |
File Type – XML Mapping – Complex # of Files = 50 parallel | 25GB x 50 | 696895 | 100% | 39% | 9h 59m (total) 8h 22m (avg/file) |
50GB x 50 | 1393690 | 100% | 39% | 20h 2m (total) 15h 53m (avg/file) | |
File Type – TEXT Mapping – Simple 1-to-1 # of Files = 1 | 25GB x 1 | 696895 | 45% | 36% | 6h 26m |
50GB x 1 | 1393690 | 45% | 36% | 12h 40m | |
200GB x 1 | 5574760 | 45% | 36% | 51h 30m |
Test D - UI Load Testing with Concurrent Users
System Configuration
- Single node (no cluster)
- Processor = Intel Xeon CPU E5-2686 2.30GHz
- Number Of Cores = 4
- Total RAM = 16GB
- OS = Linux 64-bit
- Backend / Log Database - AWS Oracle RDS
- Kernel = 8GB
- Tool used = JMeter
Result: All tests ran successfully
Scenario | Home Page | Dashboard | Network | Partners | Transactions | Max CPU % | Max RAM % |
|---|---|---|---|---|---|---|---|
# of Partners and Transactions in Back-end DB = 1138 # of Concurrent Users = 25 # of Loops (iterations) = 50
| 4.2s | 1.1s | 0.8s | 0.4s | 0.6s | 53% | 64% |
# of Partners and Transactions in Back-end DB = 1638 # of Concurrent Users = 25 # of Loops (iterations) = 50
| 4.3s | 1.0s | 0.9s | 0.3s | 0.4s | 49% | 59% |
# of Partners and Transactions in Back-end DB = 2200 # of Concurrent Users = 25 # of Loops (iterations) = 50
| 1.8s | Overview page : 3s | 6s | 0.52s | 0.62s | 44% | 60% |
| Transaction page: 0.26s | |||||||
| Trigger page: 0.33s |
Test E - Advanced Positional File
System Configuration
Single node (no cluster)
Processor = Intel Xeon CPU E5-2686 2.30GHz
Number Of Cores = 4
Total RAM =64GB
OS = Windows Server 2012 R2 64-bit
Backend / Log Database - AWS Oracle RDS
Kernel = 20GB (minimum), 40GB (maximum)
Result - All tests ran successfully
Scenario | Number of Records per file | File Size x # of Files | Total Time |
|---|---|---|---|
File Type – Text to Advanced Positional Mapping – Simple 1-to-1 | 3 million | 234 Mb x 1 | 45m 39s |
File Type – Text to Advanced Positional Mapping – Simple 1-to-1 | 5 million | 391 Mb x 1 | 1h 19m 31s |
File Type – Text to Advanced Positional Mapping – Simple 1-to-1 | 10 million | 782 Mb x 1 | 2h 31m 49s |
File Type – Text to Advanced Positional Mapping – Simple 1-to-1 | 10 million | 782 Mb x 3 | 2h 13m 4s |
File Type – Text to Advanced Positional Mapping – Simple 1-to-1 | 10 million | 782 Mb x 5 | 2h 12m 37s |
File Type – Text to Advanced Positional Mapping – Simple 1-to-1 | 10 million | 782 Mb x 10 | 2h 42m 45s |
File Type – Text to Advanced Positional Mapping – Simple 1-to-1 | 10 million | 782 Mb x 15 | 3h 56m 36s |