How StorageGRID Webscale implements the S3 REST API

A client application can use S3 REST API calls to connect to Storage Nodes and API Gateway Nodes, to create buckets, and to store and retrieve objects.

To manage these objects, the StorageGRID Webscale system uses information lifecycle management (ILM) rules.

For information about ILM rules, see the Administrator Guide.

Conflicting client requests

Conflicting client requests, such as a two clients writing to the same key, are resolved on a "latest-wins" basis. The timing for the "latest-wins" evaluation is based on when the StorageGRID Webscale system completes a given request, and not on when S3 clients begin an operation.

Consistency guarantees and controls

By default, StorageGRID Webscale guarantees read-after-write consistency for newly created objects. Any GET following a successfully completed PUT will be able to read the newly written data. Overwrites of existing objects, metadata updates, and deletes are eventually consistent. Overwrites generally take seconds or minutes to propagate, but can take up to 15 days.

StorageGRID Webscale allows you to control the consistency control used for each bucket or API request. You can change the consistency control to make a trade-off between the availability of the objects and the consistency of those objects across different Storage Nodes and sites, as required by your application.

For information on configuring consistency controls for buckets, see GET Bucket consistency request and PUT Bucket consistency request.

To set the consistency control for an individual API operation, consistency controls must be supported for the operation, and you must specify the consistency control in the request header. This example sets the consistency control to strong-site for a GET Object operation.

GET /bucket/object HTTP/1.1
Date: Sat, 29 Nov 2015 01:02:17 GMT
Authorization: AWS 9MOYPG9ACWPAJA1S72R5:jUGbYkLdBApjCWBgK4TxvOjfock=
Host: test.com
Consistency-Control: strong-site

You can set the Consistency-Control header to one of the following values:

The following behavior is applied to operations on the bucket:

Note: You must use the same StorageGRID Webscale Consistency-Control header for both the PUT Object and GET Object operations. For example, using weak to write an object and then using strong-global to read the same object back does not provide strong consistency across all sites.