Considerations for compliance

Make sure you review the considerations for using the global Compliance setting as well as the restrictions StorageGRID Webscale places on compliant buckets, compliant objects, and compliant ILM rules and policies.

Considerations for using the global Compliance setting

Restrictions for using compliant buckets

Restrictions for objects in compliant buckets

Each object that is saved in a compliant bucket goes through three stages:

  1. Object ingest
    • When an object is ingested, the system generates metadata for the object that includes a unique object identifier (UUID) and the ingest date and time. The object inherits the compliance settings from the bucket.
    • After an object is ingested into a compliant bucket, its data, S3 user-defined metadata, or S3 object tags cannot be modified, even after the retention period expires.
    • StorageGRID Webscale maintains three copies of all object metadata at each site to provide redundancy and protect object metadata from loss. Metadata is stored independently of object data.
  2. Retention period
    • The retention period for an object starts when the object is ingested into the bucket.
    • Each time the object is accessed or looked up, the compliance settings for the bucket are also looked up. The system uses the object's ingest time and date and the bucket's retention period setting to calculate when the object's retention period will expire.
    • During an object's retention period, multiple copies of the object are stored by StorageGRID Webscale. The exact number and type of copies and the storage locations are determined by the compliant rules in the active ILM policy.
      Note: As required, you might need to add new ILM rules to manage the objects in a particular bucket.
    • During an object's retention period, or when legal hold is enabled for the bucket, the object cannot be deleted.

  3. Object deletion
    • When an object's retention period ends, all copies of the object can be deleted, unless legal hold is enabled for the bucket.
    • When an object’s retention period ends, a bucket-level compliance setting allows tenant users to control how objects are deleted: by users when required or automatically by the system.
    • If the bucket setting is to delete objects automatically, all copies of the object are removed by the scanning ILM process in StorageGRID Webscale. When an object’s retention period ends, the object is scheduled for deletion. You can look for the IDEL (ILM Initiated Delete) message in the audit log to determine when ILM has started the process of auto-deleting an object because its retention period has expired (assuming the auto-delete setting is enabled and legal hold is off).
      Note: The actual amount of time needed to delete all object copies can vary, depending on the number of objects in the grid and how busy the grid processes are.

Restrictions for compliant ILM rules

If you want to enable the global Compliance setting, you must ensure that the default rule in your active ILM policy is compliant. A compliant rule satisfies the requirements of compliant S3 buckets:
  • It must create at least two replicated object copies or one erasure-coded copy.
  • These copies must exist on Storage Nodes for the entire duration of each line in the placement instructions.
  • Object copies cannot be saved on Archive Nodes.
  • At least one line of the placement instructions must start at day 0, using Ingest Time as the reference time.
  • At least one line of the placement instructions must be "forever." The actual meaning of "forever" is determined by the compliance settings for each bucket.
For example, this rule satisfies the requirements of compliant S3 buckets. It stores three replicated object copies from Ingest Time (day 0) to "forever." The objects will be stored on Storage Nodes at three data centers.
screenshot of compliant rule: 3 replicated copies from Day 0 to Forever
Note: The Make 2 Copies stock rule is compliant. You can use it as the default rule in a compliant policy.

When you configure the placement instructions for a compliant rule, you must consider where the object copies will be stored. For example, if your deployment includes more than one site, you can enable site-loss protection for compliant objects by creating a storage pool for each site and specifying both storage pools in the rule's placement instructions. See "Using multiple storage pools for cross-site replication."

Restrictions for active and proposed ILM policies

When the global Compliance setting is enabled, active and proposed ILM policies can include both compliant and non-compliant rules.
  • The default rule in the active or any proposed ILM policy must be compliant.
  • Non-compliant rules only apply to objects in non-compliant buckets.
  • Compliant rules can apply to objects in any compliant or non-compliant bucket.
As illustrated in "Example: Using compliant ILM rules in an ILM policy," a compliant ILM policy might include these three rules:
  1. A compliant rule that creates erasure-coded copies of the objects in a specific compliant S3 bucket. The EC copies are stored on Storage Nodes from day 0 to forever.
  2. A non-compliant rule that creates two replicated object copies on Storage Nodes for a year and then moves one object copy to Archive Nodes and stores that copy forever. This rule only applies to non-compliant buckets because it stores only one object copy forever and it uses Archive Nodes.
  3. A default, compliant rule that creates two replicated object copies on Storage Nodes from day 0 to forever. This rule applies to any object in any compliant or non-compliant bucket that was not filtered out by the first two rules.