What platform services are

StorageGRID Webscale platform services can help you implement a hybrid cloud strategy.

If the use of platform services is allowed for your tenant account, you can configure the following services for any S3 bucket:

Because the target location for platform services is typically external to your StorageGRID Webscale deployment, platform services give you the power and flexibility that comes from using external storage resources, notification services, and search or analysis services for your data.

Any combination of platform services can be configured for a single S3 bucket. For example, you could configure both the CloudMirror service and notifications on a StorageGRID Webscale S3 bucket so that you can mirror specific objects to the AWS Simple Storage Service™, while sending a notification about each such object to a third party monitoring application to help you track your AWS expenses.

Attention: The use of platform services must be enabled for each tenant account by a StorageGRID Webscale administrator using the Grid Management Interface or Grid Management API API. Tenant accounts that were created in versions of StorageGRID Webscale earlier than 11.0 have platform services disabled by default. Contact your grid administrator for more information.

How platform services are configured

Platform services communicate with external endpoints that you configure using the Tenant Management Interface or Tenant Management API. Each endpoint represents an external destination, such as a StorageGRID Webscale S3 bucket, an Amazon Web Services bucket, a Simple Notification Service topic, or an Elasticsearch cluster hosted locally or on AWS.

After you create an endpoint, you can enable a platform service for a bucket by adding XML configuration to the bucket. The XML configuration identifies the objects that the bucket should act on, the action that the bucket should take, and the endpoint that the bucket should use for the service.

You must add separate XML configurations for each platform service that you want to configure. For example, if you want all objects whose keys start with "/images" to be replicated to an AWS S3 bucket, you must add a replication configuration to the source bucket. If you also want to send notifications when these objects are stored to the bucket, you must add a notifications configuration. Finally, if you want to index their object metadata, you would also add the metadata notification configuration that is used to implement search integration.

The format for the configuration XML is governed by the S3 REST APIs used to implement each StorageGRID Webscale platform service:

Platform service S3 REST API
CloudMirror replication S3 bucket replication
Notifications S3 bucket notification
Search integration S3 bucket metadata notification

(custom to StorageGRID Webscale)

See the S3 Implementation Guide for more information on how StorageGRID Webscale implements these APIs.

How platform service messages are delivered

When you perform an operation on a bucket that is configured to trigger a platform services message, the message is generated and sent when the operation succeeds. For example, if a bucket is configured for CloudMirror replication, when you successfully store an object in the source bucket, a copy of that object is created and sent to the destination bucket. Delivery is asynchronous. Platform services messages are queued for delivery.

If queues grow too long or if an internal StorageGRID Webscale service responsible for platform services messages fails, operations on the source bucket fail. This failure prevents further undeliverable messages from being generated. If the destination encounters an issue that prevents it from accepting platform services messages — for example, if credentials are updated on the destination such that StorageGRID Webscale can no longer authenticate to the destination service — the operation on the bucket succeeds, but the platform services message is not delivered. In this circumstance, the grid administrator will see a Total Events (SMTT) alarm with an error message that explains the nature of the issue.

All platform services operations are performed on a per-site basis. That is, if you perform an S3 API Create operation on an object by connecting to an API Gateway Node at Data Center Site 1, the notification about that action will be triggered and sent from Data Center Site 1.

Diagram showing that action on object at site 1 triggers notification from site 1
If you subsequently perform an S3 API Delete operation on that same object from Data Center Site 2, the notification about the delete action is triggered and sent from Data Center Site 2.
diagram showing action on object from site 2 triggers notifications from site 2
Because CloudMirror replications, notifications, and search integration data are sent directly from the site where an operation is performed, a grid administrator must configure networking for each site such that messages can be delivered to the destination services.