Administration Guides

Golden Copy Overview

Home

Overview

This product offers high speed file to object copy, and sync of files to native object store over the S3 protocol.

What's New

  1. The latest releases enables
    1. Onefs 8.1.2 or later changelist API folder rename support to ensure renames are synced to the object store in the correct path
    2. 2 factor login over ssh
    3. post job stats steps recomputes job view stats
    4. Incremental sync jobs auto retry failed files
    5. Coming soon
      1. QOS full vs incremental jobs - will prioritize one of the other. Global setting applies to all folders



Deployment Diagram



Architecture



Licensing

  1. The Golden Copy tool is a cluster based license with unlimited node count and unlimited copy size of data.
  2. The license key is locked to a cluster and cannot be moved to a different cluster once the license has been applied to a cluster. An unlock license key can be purchased if this is required to move a license.

Target Use Cases:

    Synced Backup Copy of Data with 1 version of a file

    1. Requires:  Base product license, recommended to use Backup bundle advanced license key for additional reporting and monitoring.
    2. Only 1 copy or version of a file, file modifications will overwrite the previous version of the object with the new object
    3. Higher probability of recalling data
    4. Storage tier is near line and assumes data recall is important with shortest possible time to recall data
    5. Deleted files are either synced as deleted objects OR turn off deletes and retain all deleted data inn the S3 storage
    6. No expiry set on the S3 target, each version of a file is retained
    7. Full copy followed by incremental schedule using Isilon change list api and snapshots for incremental always backup.

    Synced Backup Copy of Data with N versions of a file

    1. Requires:  Backup Bundle or advanced license key
    2. Same as the above use case but allows S3 bucket versioning to be enabled.  Specify how many versions of a file should be available at any one time.
    3. Example daily incremental and store 7 versions of a file provides 7 day restore window of data.  The 8th day backup will automatically purge the oldest version of the file with a 7 version S3 bucket policy.
    4. NOTE: Versions consume the file size of the file. 
    5. Object retention expiry would not be used in this use case.
    6. Version aware restore/recall of data is supported with the backup bundle version of Golden Copy and allows an older than or newer than flag to pick which version of the day you want to recall.  Example restore data and version from Wednesday would select the closest match to Wednesday backup on a per object basis.

    One Time copy of data for long term storage or archive

    1. Requires:  The base product license
    2. Full copy of a path to long term object storage
    3. Low probability of recalling data
    4. Optional
      1. object lock retention applied
      2. tiering objects to low cost storage tier for long term retention

    Isilon/PowerScale Snapshot copy and Archive

    1. Requires: Base product license
    2. One time copy of a pre-existing snapshot for long term storage
    3. Expiry of objects will not be used on the S3 bucket in this solution

    Full backup Mode

    1. Requires:  the backup bundle or advanced license key
    2. Specify the number of full copies of a folder, the full backup creates a folder on the S3 target for each full backup copy.
    3. Object expiry would be set to the number of days each full backup should be retained before it is deleted.
    4. NOTE:  This solution requires N full copies of the folder space in the target S3 storage but offers a full backup of all data
    5. Example
      1. folder X  - 2 full monthly backups and each backup should be retained for 60 days, will ensure that 2 full backups exist at any one time.  The oldest backup would be purged by the object expiry on the S3 bucket

    Key Features

    1. A PowerScale Integrated tool to sync a copy of a path(s) on PowerScale to an S3 storage bucket.

      1. Uses PowerScale snapshot change list to support fast incremental syncs.

    2. Direct Restore - Restore Data from S3 to PowerScale and re-apply file meta data (owner, ACL's, group, data stamps). PowerScale file structure is restored directly from S3 storage on to the file system without an out of band copy.

      1. Redirected recall to a different cluster

      2. Recall to a different folder path

      3. Recall based on modified date range

    3. Protects file metadata automatically with S3 metadata tags (ACL's, owner, group, data stamps etc).

    4. MD5 checksum support for data integrity copies

    5. File system to object store audit jobs

    6. Bandwidth rate limiting per copy job

    7. Sync mode or copy mode or both

    8. Delayed Deletes to S3 copy or no deletes on source propagated to the target storage bucket - Recycle mode puts deleted files found during incremental sync and copies them to a special bucket to store deleted files. Setting a TTL on this bucket allows automatic purging on a differed schedule after files are deleted.


    © Superna LLC