Managing objects with information lifecycle management

You use information lifecycle management (ILM) to control the placement, duration, and data protection for all objects in your StorageGRID system. ILM rules determine how StorageGRID stores objects over time. You configure one or more ILM rules and then add them to an ILM policy.

ILM rules define:
  • Which objects should be stored. A rule can apply to all objects, to objects belonging to a specific tenant account or bucket/container, or to objects that contain specific metadata values.
  • The storage type and location. Objects can be stored on Storage Nodes, in Cloud Storage Pools, or on Archive Nodes.
  • The type of object copies made. Copies can be replicated or erasure coded.
  • For replicated copies, the number of copies made.
  • For erasure coded copies, the erasure coding algorithm used.
  • The changes over time to an object's storage location and type of copies.
  • How object data is protected as objects are ingested into the grid (synchronous placement or dual commit).

Note that object metadata is not managed by ILM rules. Instead, object metadata is stored in a Cassandra database in what is known as a metadata store. Three copies of object metadata are automatically maintained at each site to protect the data from loss. The copies are load balanced across all Storage Nodes.

Example ILM rule

This example ILM rule applies to all objects belonging to Tenant A. It makes three replicated copies of those objects and stores each copy at a different data center for 2 years. After 2 years, one copy is moved to archive storage "forever."

This rule uses the Balanced option for ingest behavior: the three-copy placement instruction is applied as soon as Tenant A saves an object to StorageGRID, unless it is not possible to immediately make all three required copies. In that case StorageGRID immediately makes two interim copies on two different Storage Nodes, and makes the required copies later.

Example Rule for 3 Copies

How an ILM policy evaluates objects

The active ILM policy for your StorageGRID system controls the placement, duration, and data protection of all objects.

When clients save objects to StorageGRID, the objects are evaluated against the ordered set of ILM rules in the active policy, as follows:
  1. If the filtering criteria for the first rule in the policy match an object's metadata, the object is ingested according to that rule's ingest behavior and stored according to that rule's placement instructions.
  2. If the filtering criteria for the first rule does not match the object's metadata, the object is evaluated against each subsequent rule in the policy until a match is made.
  3. If no rules match the object's metadata, the ingest behavior and placement instructions for the default rule in the policy are applied.

Example ILM policy

This example ILM policy uses three ILM rules.
Example Policy for Two Tenants

In this example, all objects belonging to Tenant A are matched by the first rule and are stored as three replicated copies at three data centers. Objects belonging to other tenants are not matched by the first rule, so they are evaluated against the next rule in the policy.

The second ILM rule applies to objects stored by any tenant that are larger than 200 KB. These larger objects are stored using erasure coding across two data center sites. Objects 200 KB or smaller are not matched by the second rule, so they are evaluated against the third rule.

The third rule is the default rule for the policy. The default rule is applied to any objects that do not match any other rule in the policy. In this example, the default rule makes two replicated copies of all objects 200 KB or smaller that do not belong to Tenant A.

Policy and Rules