Make sure you review the considerations for using the global Compliance setting as well as the restrictions StorageGRID Webscale places on compliant buckets, compliant objects, and compliant ILM rules and policies.
Considerations for using the global Compliance setting
- You must enable the global Compliance setting before any S3 tenant can create a compliant bucket.
- After you enable the global Compliance setting, you cannot disable this setting.
- Enabling the global Compliance setting allows all S3 tenant accounts to use the Tenant Manager, the Tenant Management API, or the S3 REST API to create and manage compliant buckets. Users with the appropriate permissions can create compliant buckets, set and increase the retention period for objects in the bucket, specify how objects can be deleted at the end of their retention period, and optionally place all objects in the bucket under a legal hold or lift a legal hold.
For example, this tenant user is creating a compliant bucket named bank-records in the default us-east-1 region. Objects in this bucket will be retained for 6 years and then deleted automatically. This bucket is not currently under a legal hold.
- When the global Compliance setting is enabled, you cannot create a new proposed ILM policy or activate an existing proposed ILM policy unless the default rule in the policy satisfies the requirements of S3 compliant buckets. The ILM Rules and ILM Policies pages indicate which ILM rules are compliant.
In the following example, the ILM Rules page lists two rules that are compatible with compliant buckets.
Restrictions for using compliant buckets
- If S3 tenants need to create compliant buckets, they must enable compliance and specify compliance settings when they create the bucket. After a bucket has been saved, compliance cannot be disabled for the bucket.
- The retention period for the bucket specifies the minimum amount of time each object in that bucket must be preserved (stored) within StorageGRID Webscale.
- Tenant users can edit bucket settings to increase the retention period, but they can never decrease this value.
- If a tenant account is notified of a pending legal action or regulatory investigation, users can preserve relevant information by placing a legal hold on the bucket. When a bucket is under a legal hold, no object in that bucket can be deleted even if its retention period has ended. As soon as the legal hold is lifted, objects in the bucket can be deleted when their retention periods end.
- Objects can be added to a compliant bucket at any time, regardless of the bucket's compliance settings.
- Objects can be retrieved from a compliant bucket at any time, regardless of the bucket's compliance settings.
- Versioning is not supported for compliant buckets.
Restrictions for objects in compliant buckets
Each object that is saved in a compliant bucket goes through three stages:
- Object ingest
- When an object is ingested, the system generates metadata for the object that includes a unique object identifier (UUID) and the ingest date and time. The object inherits the compliance settings from the bucket.
- After an object is ingested into a compliant bucket, its data, S3 user-defined metadata, or S3 object tags cannot be modified, even after the retention period expires.
- StorageGRID Webscale maintains three copies of all object metadata at each site to provide redundancy and protect object metadata from loss. Metadata is stored independently of object data.
- Retention period
- The retention period for an object starts when the object is ingested into the bucket.
- Each time the object is accessed or looked up, the compliance settings for the bucket are also looked up. The system uses the object's ingest time and date and the bucket's retention period setting to calculate when the object's retention period will expire.
- During an object's retention period, multiple copies of the object are stored by StorageGRID Webscale. The exact number and type of copies and the storage locations are determined by the compliant rules in the active ILM policy.Note: As required, you might need to add new ILM rules to manage the objects in a particular bucket.
-
During an object's retention period, or when legal hold is enabled for the bucket, the object cannot be deleted.
- Object deletion
- When an object's retention period ends, all copies of the object can be deleted, unless legal hold is enabled for the bucket.
- When an object’s retention period ends, a bucket-level compliance setting allows tenant users to control how objects are deleted: by users when required or automatically by the system.
- If the bucket setting is to delete objects automatically, all copies of the object are removed by the scanning ILM process in StorageGRID Webscale. When an object’s retention period ends, the object is scheduled for deletion. You can look for the IDEL (ILM Initiated Delete) message in the audit log to determine when ILM has started the process of auto-deleting an object because its retention period has expired (assuming the auto-delete setting is enabled and legal hold is off). Note: The actual amount of time needed to delete all object copies can vary, depending on the number of objects in the grid and how busy the grid processes are.
Restrictions for compliant ILM rules
If you want to enable the global Compliance setting, you must ensure that the default rule in your active ILM policy is compliant. A compliant rule satisfies the requirements of compliant S3 buckets:
- It must create at least two replicated object copies or one erasure-coded copy.
- These copies must exist on Storage Nodes for the entire duration of each line in the placement instructions.
- Object copies cannot be saved on Archive Nodes.
- At least one line of the placement instructions must start at day 0, using Ingest Time as the reference time.
- At least one line of the placement instructions must be "forever." The actual meaning of "forever" is determined by the compliance settings for each bucket.
For example, this rule satisfies the requirements of compliant S3 buckets. It stores three replicated object copies from Ingest Time (day 0) to "forever." The objects will be stored on Storage Nodes at three data centers.
Note: The Make 2 Copies stock rule is compliant. You can use it as the default rule in a compliant policy.
When you configure the placement instructions for a compliant rule, you must consider where the object copies will be stored. For example, if your deployment includes more than one site, you can enable site-loss protection for compliant objects by creating a storage pool for each site and specifying both storage pools in the rule's placement instructions. See "Using multiple storage pools for cross-site replication."
Restrictions for active and proposed ILM policies
When the global Compliance setting is enabled, active and proposed ILM policies can include both compliant and non-compliant rules.
- The default rule in the active or any proposed ILM policy must be compliant.
- Non-compliant rules only apply to objects in non-compliant buckets.
- Compliant rules can apply to objects in any compliant or non-compliant bucket.
As illustrated in "Example: Using compliant ILM rules in an ILM policy," a compliant ILM policy might include these three rules:
- A compliant rule that creates erasure-coded copies of the objects in a specific compliant S3 bucket. The EC copies are stored on Storage Nodes from day 0 to forever.
- A non-compliant rule that creates two replicated object copies on Storage Nodes for a year and then moves one object copy to Archive Nodes and stores that copy forever. This rule only applies to non-compliant buckets because it stores only one object copy forever and it uses Archive Nodes.
- A default, compliant rule that creates two replicated object copies on Storage Nodes from day 0 to forever. This rule applies to any object in any compliant or non-compliant bucket that was not filtered out by the first two rules.