Product Requirements, Cluster Sizing and Tested Scaling Limits
Minimum VM Specifications
- Read and write latency average latency per vm < 20 ms
- ecactl cluster exec iostat -xyz -d 3
- 4 x vCPU per vm
- See how much disk space to add in the examples below, match to the closest largest matched configuration.
- * Note this is starting disk size, content indexing and metadata indexing will require adding more storage as more files are indexed. If disk space utilization reaches 70% all indexing will stop automatically. The above are only starting disk space requirements. Additional RAM is required as per below
- ** Content indexing or high file count or quick reports that run against a high file count requires more RAM per VM 20G is the minimum and file count will determine total ram per node. Heap usage per node must be below 75% used. If Heap usage rises additional RAM will be required for each node. Heap usage can be viewed on node one https://x.x.x.x/solr then select solr cloud
- *** Indexing rate is directly dependent on read latency first and then write latency 2nd. If read latency is above specified values, indexing rate will drop as a factor of the read latency to the disks in the VM.
- **** Content indexing - It is expected to add disk space to the index as different content types, and numbers of files that require content ingestion will increase the index size. This is normal, expected, and supported to add disk space online during indexing operations. Content type and volume of content indexing differences does not allow prediction of disk space required. Rough estimates for planning purposes should assume 10%-20% of the original data size. File formats vary greatly in how much text vs formatting or images a file contains.
Sample configurations based on file count
NOTE: These are guidelines only and not a guarantee disk space requirement.
NOTE: Content indexing is RAM heavy operation.
If you do not see a match to your use case contact sales@superna.net
| Configuration | VM count | Total Disk Space | HA Mode | RAM per VM |
| 250 Million files Meta data reporting | 4 | 430 GB per VM = 1720 GB (base OVA disk space) | If HA mode is disabled divide the disk space requirement by 50% | 16GB (base OVA RAM ) |
| 500 Million files Meta data reporting | 4 | 430 GB per VM = 1720 GB (base OVA disk space) | If HA mode is disabled divide the disk space requirement by 50% | 24GB (Add 8GB RAM VM 2-4 ) |
| 750 Million files Meta data reporting | 7 | 630 GB per VM = 4210 GB (Add 200 GB to VM 2-7) | If HA mode is disabled divide the disk space requirement by 50% | 24GB (Add 8GB RAM VM 2-7 ) |
| 1B files Meta data reporting | 7 | 830 GB per VM = 5410 GB (Add 400 GB to VM 2-7) | If HA mode is disabled divide the disk space requirement by 50% | 30GB (Add 14GB RAM VM 2-7 ) |
2B files Meta data reporting | 7 | 1230 GB per VM = 7810 GB (Add 800 GB to VM 2-7) | If HA mode is disabled divide the disk space requirement by 50% | 36GB (Add 20GB RAM VM 2-7 ) |
3B files Meta data reporting | 7 | 1730 GB per VM = 12110 GB (Add 1300 GB to VM 2-7) | If HA mode is disabled divide the disk space requirement by 50% | 40GB (Add 24GB RAM VM 2-7 ) |
4B + files Meta data reporting | 10 | 1730 GB per VM = 13840 GB (Add 1300 GB to VM 2-10) | If HA mode is disabled divide the disk space requirement by 50% | 40GB (Add 24GB RAM VM 2-10 ) |
500 Million files content indexing (this means 500 million files that are content indexed) Data types determines disk space this is guideline only and actual indexing will determine disk expansion requirements | 7 | 1660 GB per VM = 10390 GB (Add 1230 GB to VM 2-7) | If HA mode is disabled divide the disk space requirement by 50% | 40GB (Add 24GB RAM VM 2-7 ) |
| 250 Million files content indexing (this means 250 million files that are content indexed) Data types determines disk space this is guideline only and actual indexing will determine disk expansion requirements | 7 | 800 GB per VM = 5230 GB (Add 370 GB to VM 2-7) | If HA mode is disabled divide the disk space requirement by 50% | 40GB (Add 24GB RAM VM 2-7 ) |
Feature Tested Limits
The following are tested limits. These do not represent actual limits. These numbers will be updated with future releases.
| Scaling Limit Item | Tested Value |
| Number of files or directories meta data only index (requires 7-9 VM configuration) | 12 billion |
| Maximum file size for content Indexing | 500 MB |
| Number of files in a directory | 1 Million |
| Number of subdirectories in a path | 1 Million |
| Number of paths added for indexing | 25 |
| Number of clusters added to single appliance for indexing | 2 |
| Number Snapshots to Monitor (1.1.5) | 25 |
© Superna Inc