Administration Guides

Understanding Copy Performance Configurations

Home



Overview

This section should be reviewed when deciding how to configure which Golden Copy for copying files, and which IP pool interfaces will be used to copy files.   In addition, decisions on parallel copy influences the speed of copies and the bandwidth consumed for a copy job.  Golden Copy also supports multipart uploads and downloads. 

How to scale out and scale up copy performance

  1. Deploy Virtual Accelerator nodes to increase copy performance.  The configuration is 1 VM or 6 VM's.  The main Golden Copy VM is the control VM and the remaining 5 VM's are only used for copying.    All VM's must have an NFS mount to the source cluster.
    1. See the install guide to deploy VAN nodes here.
  2. Parallel threads can be increased to use more threads on multi part uploads.  The default is 10 threads.
    1. This can be changed following steps below.
    2. ssh to node 1 as ecaadmin
    3. nano /opt/superna/eca/eca-env-common.conf
    4. Add this tag to double the threads on large files.  This should only be used if the majority of files are over 1GB in size.
    5. export ARCHIVE_PARALLEL_THREAD_SDK=20
    6. control+x , answer yes to save
    7. ecactl cluster down 
    8. ecactl cluster up


Dell ECS Performance Copy Mode

  1. Golden Copy supports a load balance option with health check of ECS nodes + load balancing copies evenly to ECS nodes provided. 
  2. This will maximize performance and eliminates the requirement to use an external load balcner that adds cost to the overall solution.

How load balancing works

  1. Each Golden Copy node uses a S3 health check to each ECS node, this is performed every minute. If an ECS node is unreachable or fails the S3 health check it is removed from the pool of ECS nodes.  
  2. If the ECS nodes passes the S3 health check in the next 1 minute, it will be added back to the load balancing pool.  The algorithm will ensure a balanced number of copies per ECS node using the parallel thread setting on the Golden copy vm.
© Superna LLC