Considerations for using platform services

Before implementing platform services, review the recommendations and considerations for using these services.

Considerations for using platform services

Consideration Details

Destination endpoint monitoring

You must monitor the availability of each destination endpoint. If connectivity to the destination endpoint is lost for an extended period of time and a large backlog of requests exists, additional client requests (such as PUT requests) to StorageGRID Webscale will fail. You must retry these failed requests when the endpoint becomes reachable.

Destination endpoint throttling

StorageGRID Webscale might throttle incoming S3 requests for a bucket if the rate at which the requests are being sent exceeds the rate at which the destination endpoint can receive the requests. Throttling only occurs when there is a backlog of requests waiting to be sent to the destination endpoint.

The only visible effect is that the incoming S3 requests will take longer to execute. If you start to detect significantly slower performance, you should reduce the ingest rate or use an endpoint with higher capacity. If the backlog of requests continues to grow, client S3 operations (such as PUT requests) will eventually fail.

CloudMirror requests are more likely to be affected by the performance of the destination endpoint because these requests typically involve more data transfer than search integration or event notification requests.

Ordering guarantees

StorageGRID Webscale makes a best effort attempt to order requests to the destination endpoint. Ordering of requests is not guaranteed when operations are made across StorageGRID Webscale sites. For example, if you write an object initially to site A and then later overwrite the same object at site B, the final object replicated by CloudMirror to the destination bucket is not guaranteed to be the newer object.

ILM-driven object deletions

To match the deletion behavior of the AWS CRR and SNS services, CloudMirror and event notification requests are not sent when an object in the source bucket is deleted because of StorageGRID Webscale ILM rules. For example, no CloudMirror or event notifications requests are sent if an ILM rule deletes an object after 14 days.

In contrast, search integration requests are sent when objects are deleted because of ILM.

Considerations for using the CloudMirror replication service

Consideration Details

Replication status

StorageGRID Webscale does not support the x-amz-replication-status header.

Bucket versioning

If the source S3 bucket in StorageGRID Webscale has versioning enabled, you should also enable versioning for the destination bucket.

When using versioning, note that the ordering of object versions in the destination bucket is best effort and not guaranteed by the CloudMirror service, due to limitations in the S3 protocol. Version IDs for the source bucket in StorageGRID Webscale are not related to the version IDs for the destination bucket.

Tagging for object versions

The CloudMirror service does not replicate any PUT Object tagging or DELETE Object tagging requests that supply a version ID, due to limitations in the S3 protocol. Because version IDs for the source and destination are not related, there is no way to ensure that a tag update to a specific version ID will be replicated.

In contrast, the CloudMirror service does replicate PUT Object tagging requests or DELETE Object tagging requests that do not specify a version ID. These requests update the tags for the latest key (or the latest version if the bucket is versioned). Normal ingests with tags (not tagging updates) are also replicated.