Administration Guides

Limitations, Requirements and Supported Target List

Home


Firewall Port Requirements:

  1. SSH and HTTPS 8080 REST API from Golden Copy to PowerScale IP pool in the System Zone.
  2. S3 port 443 https to target S3 storage.
  3. NFS access from Golden Copy and Virtual Accelerator Nodes to Systems zone IP pool
  4. Full firewall port table and directions can be found here.

Supported Browsers

  1. Windows Chrome, Edge (Chromium only)
    1. No browser plugs are supported if they interfere with UI functions, customers must disable them or contact the vendor for assitance.

Product Limitations and S3 Target Requirements

  1. Data copied to the bucket using tools other than Golden Copy will be recalled but metadata will not be recalled for this data.   The data must exist under the Golden Copy path in the bucket  <cluster name>/ifs/xxxxx   if the data is copied under this object path it will be recalled without metadata.  (Requires 1.1.6 or later)
  2. File Size differences when viewing files over S3 tools and Onefs using pipeline or backup and sync modes is normal.  Onefs Display application and logical size and S3 will display bytes only.    You can verify this with md5 checksum but it is common to see files copied or synced that will display different file sizes.  md5 checksum can be used to verify but it is normal and expected difference between onefs file system display of size vs S3.
  3. Active Directory resolution must be configured on Powerscale to resolve UID and GID, values on files.
  4. Symlink files are skipped and will not be copied
  5. hardlink files are copied
  6. Fast Incremental performance requires OneFs 8.2.1 or later releases.  The change list API includes metadata and removes the requirement to query the file system for metadata of the files in the change list.
    1. Releases before 8.2.1 do not support change list + metadata, this requires additional query to the file system for each file in the change list this will impact performance of processing a large change list of files.
    2. Solution is to upgrade to 8.2.1 of Onefs
  7. (Solved with unique long path support feature) Copying paths longer than 1001 characters due to REST API limitation on characters per request. This means files at folders below this level will not be copied to the S3 bucket the path is calculated /ifs/.snapshot/.... path to file and file name.   NOTE:  Windows SMB path length limit is 260 characters.
    1. Golden copy Long path support feature.
  8. Concurrent Running jobs - Releases above 1.1.4 21073 or later
    1. NOTE:  More concurrent jobs does not mean increased throughput.  Copy resources are shared across all running jobs.
    2. Number of folders configured for Incremental always scheduled concurrently is a maximum 80.  It is Not supported to exceed this concurrent folder incremental limit, contact support.
    3. In order to exceed 80 folders with incremental it is possible to offset additional folders to start on different hours to stay within the 80 concurrent limit.  example every 4 hours start 5 folder incremental and the next 4 hour interval start 5 different folders.
    4. Full copy concurrent testing limit of 80 jobs but only 10 jobs execute at a time with resources shared across all jobs that are running. 
  9. Report creation is limited to 15 M success files but can be extended by adding disk to the vm to support exporting copy job reports in json format to any file count required by allocating more disk space for logging successful file copies.
  10. Concurrent running jobs display is limited to 10 active running jobs in the CLI any job submitted after 10 will not be displayed but it will be queued once one of the first 10 jobs completes,  queued jobs will appear as running.
  11. Largest Tested  single file multipart upload is 1TB  file.   No architectural limit blocks larger files other than the S3 target max object size.
  12. Directories with no files stored in them will not be copied to the object store as an empty directory.  The directory will not be visible in the object store.
  13. ADS or Alternate data stream files will not have ADS streams copied
  14. Characters supported in file and directory names as per Amazon S3 specification listed here.

    1. Special Characters

      AWS - Folder name with special chars

      AWS - File name with special chars

      ECS - Folder name with special chars

      ECS - File name with special chars

      ~

      pass

      pass

      pass

      pass

      `

      pass

      pass

      pass

      pass

      !

      pass

      pass

      pass

      pass

      @

      pass

      pass

      pass

      pass

      #

      pass

      pass

      pass

      pass

      $

      pass

      pass

      pass

      pass

      %

      pass

      pass

      pass

      pass

      ^

      pass

      pass

      pass

      pass

      &

      pass

      pass

      pass

      pass

      ()

      pass

      pass

      pass

      pass

      _

      pass

      pass

      pass

      pass

      +

      pass

      pass

      pass

      pass

      -

      pass

      pass

      pass

      pass

      =

      pass

      pass

      pass

      pass

      []

      pass

      pass

      pass

      pass

      {}

      pass

      pass

      pass

      pass

      :

      pass

      pass

      pass

      pass

      ;

      pass

      pass

      pass

      pass

      '

      pass

      pass

      pass

      pass

      ,

      pass

      pass

      pass

      pass

      .

      pass

      pass

      pass

      pass

      \

      Not supported

      Not supported

      Not supported

      Not supported

      *

      pass

      pass

      pass

      pass

      ?

      pass

      pass

      Not supported

      Not supported

      <>

      pass

      pass

      pass

      pass

      "

      Not supported

      Not supported

      Not supported

      Not supported

      |

      *

      pass

      pass

      pass

       
    2. Windows does not support these characters in file or folder names \ / : * ? " < > |
    3. Azure does not support:   \ %  
    4. Language characters have been tested successfully for non ascii characters in file names or folders.  Not all languages or characters have been tested.
    5. UTF-8 encoded file names and paths
    6. Not supported
      1. All other special characters not listed above
  15. Connecting to S3 targets on the Internet requires Golden Copy IP addresses to be NATed to the Internet. This applies to AWS and Azure or any other Internet S3 target.
  16. S3 Targets must support custom object properties for protecting metadata of the files and folders.
  17. Isilon or Powerscale any folder can be copied including snapshots.  In addition the DR cluster that is read-only due to synciq can also be used as a source of a folder copy.
  18. Recall  Cluster Targets
    1. Golden Copy Basic license 1 recall target configured at a one time.
    2. Golden Copy Advanced or Backup Bundle License recall target limit of 5 with the ability to select a target during a recall job creation

S3 Target Device Specific Feature Support

  1. Read Me First About Target Compatibility

    1. Customers must review all limitations of any selected target including rate limiting, billing model. All Service providers rate limit and it is customers responsibility to contact the service provider for rate limits or quotas.   The compatibility below verifies API's used by Golden Copy product functionality has been tested. No performance guarantee can be made for any target. Any billing issues as a result of using any target below with Golden Copy is a customer responsibility.  All target errors that require investigation require the customer to have valid support with the provider and will be asked to open cases regarding target storage errors.

  2. Dell ECS

    1. Compliance mode is supported
    2. Bucket Versions are supported
    3. Retention policies and versions at the bucket level in a combined policy is not supported by ECS
    4. Application Aware meta data content type encoding
    5. Checksum support
    6. Supports a load balance option with health check of ECS nodes + load balancing copies evenly to ECS nodes provided.  Typically has higher performance than using a load balancer.

  3. Amazon Amazon S3

    1. Snowball Edge
    2. AWS auth v4
    3. Transfer Acceleration Buckets. (planned future release)
    4. Client Side KMS keys - not supported - TLS inflight encryption for inflight security only.
    5. Retention policies
    6. Access and Security authentication only
    7. Application Aware meta data content type encoding
    8. Checksum support
    9. Multipart upload and download 
    10. Object Lock support

  4. Microsoft Azure Blob 

    1. Blob storage REST API
    2. Data Box
    3. Retention policies
    4. checksum support
    5. Multipart uploads
    6. Does Not support multipart downloads
    7. No content type encoding support binary/octet used for all files
    8. Object lock supported release 1.1.6 or later.

  5. Google Cloud Storage 

    1. Native SDK API
    2. Checksum support
    3. Multipart upload and download
    4. all metadata options supported

  6. Cohesity

    1. See vendor documentation for versioning support, and object retention policy support.
    2. NOTE: Does not support load balancing feature in Golden Copy due to no support for multi part copy of chunks load balanced across Cohesity nodes.
    3. checksum support
    4. Application Aware meta data content type encoding
    5. Checksum support

  7. OpenIO

    1. Versioning not tested
    2. eca-env-common.conf requires export ARCHIVE_S3_PREFIX="igls-"
    3. checksum support
    4. Application Aware meta data content type encoding
    5. Checksum support

  8. Ceph version 15 or later Octopus 

    1. aws v4 signature only
    2. checksum support
    3. Application Aware meta data content type encoding
    4. Checksum support

  9. MinIO https://min.io/

    1. aws v4 signature only
    2. eca-env-common.conf requires export ARCHIVE_S3_PREFIX="igls-"
    3. cloud type other must be used
    4. checksum support
    5. Application Aware meta data content type encoding
    6. Checksum support

  10. Scality Open Source 

    1. Versioning not tested
    2. cloud type other must be used
    3. checksum supported
    4. example add command
      1. searchctl archivedfolders add --isilon gcsource --folder /ifs/archive --accesskey accessKey1 --secretkey verySecretKey1 --endpoint http://x.x.x.x:8000 --region us-east-1 --bucket bucket --cloudtype other
        create a folder which will become the bucket name when created at the root of the target device.
    5. Application Aware meta data content type encoding
    6. Checksum support

  11. Wasabi

    1. NOTE:  Contact Wasabi support about service throughput details regarding any limitations for high performance copies.
    4. Versioning tested
    5. Checksum support
    6. cloud type other
    7. full permissions to the bucket are required
    8. special characters in metadata not supported example AD domain or owner of file with special characters not supported.  Requires base64 encoding feature to support special characters.   See support on how to configure.
    9. No orphaned multipart clean up life cycle support on buckets or abort multipart upload api support.  
    10. example command
      1. searchctl archivedfolders add --isilon gcsource --folder /ifs/archive --accesskey --secretkey --endpoint https://s3.wasabisys.com --bucket gctest --cloudtype other
    11. In order to use Wasabi, Transfer Manager V2 must be enabled. This is done by setting the variable AWS_USE_TRANSFER_MANAGER_V2 like below:

                         AWS_USE_TRANSFER_MANAGER_V2=true

  1. IBM Cloud Object Storage

    1. Versioning tested
    2. Checksum support
    3. cloud type other
    4. example command
      1. searchctl archivedfolders add --isilon gcsource --folder /ifs/archive --accesskey --secretkey --endpoint https://<endpoint> --bucket gctest --cloudtype other

  2. BackBlaze S3

    1. Versioning tested
    2. Checksum support
    3. cloud type other
    4. example command
      1. searchctl archivedfolders add --isilon gcsource --folder /ifs/archive --accesskey --secretkey --endpoint https://<endpoint> --bucket gctest --cloudtype other
    1. Versioning tested
    2. Checksum support
    3. Multi part upload
    4. cloud type AWS
    5. note: bucket url must resolve to the end point ip address using url access method.  example gctest.s3-us-east.internal.superna.net  must resolve in DNS on the Golden Copy VM's
    6. Example Command
      1. searchctl archivedfolders add --isilon gcsource --folder /ifs/archive --accesskey --secretkey  --endpoint https://s3-us-east.internal.superna.net --bucket gctest --region us-east --cloudtype aws

  4. Pure Storage FlashBlade

    1. Versioning tested
    2. Checksum support
    3. Multi part upload
    4. cloud type AWS
    5. example command
      1. searchctl archivedfolders add --isilon <cluster name> --folder <path> --accesskey yyyyy --secretkey xxxx --endpoint https://<dns name> --bucket <bucket name> --region us-east-1 --cloudtype aws 

  5. Weka S3

    1. Versioning not supported
    2. Checksum support
    3. eca-env-common.conf requires export ARCHIVE_S3_PREFIX="igls-"
    4. Multi part upload
    5. cloud type Other
    6. Advanced metadata support
    7. Cloud Browser GUI not supported
    8. example command
      1. searchctl archivedfolders add --isilon <cluster name> --folder  <path> --accesskey yyyy --secretkey xxxx --endpoint https://<dns name>:9000 --bucket <bucket name> --cloudtype other

  6. Hitachi Vantara HCP CS

    1. versioning supported
    2. checksum supported
    3. multi part upload supported
    4. multi part download supported
    5. Advanced metadata supported
    6. Cloud browser GUI supported
    7. example command
      1. searchctl archivedfolders add --isilon <cluster name> --folder <path> --accesskey xxxx --secretkey yyyyy --endpoint https://<dns name> --bucket <bucket name> --region us-east-1 --cloudtype aws

  7. Dell ObjectScale

    1. object versioning supported
    2. checksum supported
    3. multi part upload supported
    4. multi part download supported
    5. Advanced metadata supported
    6. CloudBrowser support
    7. Requires ObjectScale 1.3 or 1.2 with this change.
      1. svc_param set -n com.emc.ecs.s3.auth.verify_signature_with_encoded_uri -v false -r test 
    8. example Command to add a folder
      1. searchctl archivedfolders add --isilon <cluster name> --folder <path> --accesskey yyyyy --secretkey xxxxxx --endpoint https://c.c.c.c:443 --bucket <bucket name> --region us-east-1 --cloudtype ecs

  8. Quantum ActiveScale 

    1. object versioning supported
    2. checksum supported
    3. multi part upload supported
    4. multi part download supported
    5. Advanced metadata supported
    6. Cloud Browser support
    7. example command to add a folder
      1. searchctl archivedfolders add --isilon <cluster name> --folder '<path>' -accesskey yyyyy --secretkey xxxxxx --endpoint https://c.c.c.c:443 --bucket <bucket name> --region us-east-1 --cloudtype other

  9. Vast Data

    1. object versioning supported
    2. checksum supported
    3. multi part upload supported
    4. multi part download supported
      1. Example command to add a folder
      2. searchctl archivedfolders add --isilon gcsource --folder /ifs/archive --accesskey yyyyy --secretkey xxxxxx --endpoint https://c.c.c.c --bucket <bucket name> --region us-east-1 --cloudtype aws 

  10. Netapp Ontap

  1. object versioning supported
  2. checksum supported
  3. multi part upload supported
  4. multi part download supported
  5. Advanced metadata supported
  6. Cloud Browser support
  7. example command to add a folder
    1. searchctl archivedfolders add --isilon <cluster name> --folder '<path>' -accesskey yyyyy --secretkey xxxxxx --endpoint https://c.c.c.c --bucket <bucket name> --region us-east-1 --cloudtype aws



© Superna Inc