Creating an ILM rule

ILM rules allow you to manage the placement of object data over time. To create an ILM rule, you use the Create ILM Rule wizard.

Before you begin

About this task

When StorageGRID evaluates objects against an ILM rule, it looks first at the rule's basic and advanced filtering criteria. If an object matches the filtering criteria, the object is copied and placed according to the rule's placement instructions. Placement instructions determine where, when, and how object data is stored. If the rule includes more than one placement instruction, when a set time expires, the content placement instructions for the next time period are applied to objects at the next ILM evaluation time.

Steps

  1. Select ILM > Rules.
    The ILM Rules page appears, with the stock rule, Make 2 Copies, selected.


    ILM Rules page

    Note: If the global Compliance setting has been enabled for the StorageGRID system, the ILM Rules page indicates which ILM rules are compliant. The summary table includes a Compliant column, and the details for the selected rule include a Compliance Compatible field. See "Managing S3 buckets and objects for compliance" for more information.
  2. Click Create.
    Step 1 of the Create ILM Rule wizard appears.


    Create ILM Rule step 1 of 2

  3. Complete Step 1 of the Create ILM Rule wizard.
    1. Enter a unique name for the rule in the Name field.
      You must enter between 1 and 64 characters.
    2. Optionally, enter a short description for the rule in the Description field.
      You should describe the rule's purpose or function so you can recognize the rule later.


      Rule name and description

    3. From the Tenant Account drop-down list, optionally select the S3 or Swift tenant account to which this rule applies. If this rule applies to all tenants, select Ignore (default).
    4. Use the Bucket Name field to specify the S3 buckets or Swift containers to which this rule applies.
      If matches all is selected (default), the rule applies to all S3 buckets or Swift containers.
    5. Optionally, click Advanced filtering, and specify additional filtering criteria.
      If you do not configure advanced filtering, the rule applies to all objects that match the currently configured criteria (tenant account and bucket).
      See "Using advanced filters in ILM rules" for information about the types of metadata, operators, and metadata values you can specify.
  4. Click Next.
    Step 2 of the wizard appears.


    Create ILM Rule step 2 of 2

  5. For Reference Time, select the type of time to use when calculating the start time for a placement instruction.
    Option Description
    Ingest Time The time when the object was ingested.
    Last Access Time The time when the object was last retrieved (read or viewed).
    Note: To use this option, updates to Last Access Time must be enabled for the S3 bucket or Swift container.
    Noncurrent Time The time an object version became noncurrent because a new version was ingested and replaced it as the current version.
    Note: The Noncurrent Time applies only to S3 objects in versioning-enabled buckets.

    You can use this option to reduce the storage impact of versioned objects by filtering for noncurrent object versions updated with a new current version or delete marker.

    User Defined Creation Time A time specified in user-defined metadata.
    Note: If you want to create a compliant rule, you must select Ingest Time. See "Managing S3 buckets and objects for compliance."
  6. In the Placements section, select a starting time and a duration for the first time period.
    For example, you might want to specify where to store objects for the first year ("day 0 for 365 days"). At least one instruction must start at day 0.
    Attention: Do not specify an Archive Node or a Cloud Storage Pool as the location for the placement instruction starting on day 0. On day 0 (ingest), objects should only be stored on Storage Nodes. To store object copies in other locations, click Add to add a second time period starting on day 1 or later.
  7. If you want to create replicated copies on one or more storage pools:
    1. From the Type drop-down list, select replicated.
    2. In the Location field, optionally click Add Pool. Then, select one or more storage pools.
      If you are specifying more than one storage pool, keep these rules in mind:
      • If you are specifying more than one storage pool and creating n copies, you must add n or more pools. For example, if you plan to specify three copies, you must specify three or more storage pools.
      • If the number of copies equals the number of storage pools, one copy of the object is stored in each storage pool.
      • If the number of copies is less than the number of storage pools, the system distributes the copies to keep disk usage among the pools balanced, while ensuring that one copy goes only to one site.
      • If the storage pools overlap (contain the same Storage Nodes), all copies of the object might be saved at only one site. For this reason, do not specify the default storage pool (All Storage Nodes) and another storage pool.

      Placement instructions for multiple storage pools
    3. Select the number of copies you want to make.
      Attention: In general, do not configure ILM rules to create only one replicated copy. If the only replicated copy is lost or corrupt, data will be lost. In addition, you might lose access to the object during maintenance or recovery operations. Use ILM to create a single replicated copy only if you are willing to risk the loss of object data when a failure occurs.
      Note: StorageGRID can store only one replicated copy of an object on any given Storage Node. If your grid includes three Storage Nodes and you select 4 as the number of copies, be aware that only three copies will be made—one copy for each Storage Node.
    4. If you are using only a single storage pool, specify a temporary storage pool.

      Temporary storage pool

      Specifying a temporary storage pool is optional, but recommended. If the preferred storage pool is unavailable, a copy is made in the temporary storage pool. As soon as the preferred storage pool becomes available, a copy is made in the preferred storage pool, and the copy in the temporary storage pool is deleted.

      Attention: Failing to specify a temporary storage pool puts object data at risk if the preferred pool is unavailable.
  8. If you want to move objects to a Cloud Storage Pool:
    1. From the Type drop-down list, select replicated.
    2. In the Location field, remove All Storage Pools, and click Add Pool. Then, select the Cloud Storage Pool

      Adding a Cloud Storage Pool to a placement instruction
      If you want to use a Cloud Storage Pool, keep these rules in mind:
      • You can create only one replicated copy.
      • You cannot store the object in another storage pool or Cloud Storage Pool, or as an erasure coded copy, during the same time period.

        In the following example, the All Storage Nodes storage pool was selected from day 0 to day 365. An error message appears when a Cloud Storage Pool is selected for the same time period.


        Placement instructions for Cloud Storage Pool
  9. If you want to create an erasure-coded copy:
    1. From the Type drop-down list, select erasure coded.
      The number of copies changes to 1.
    2. Select the storage location.
      The storage locations for an erasure-coded copy include the name of the storage pool, followed by the name of the Erasure Coding profile.


      Storage pool and EC profile name

      Attention: When adding a rule that makes an erasure-coded copy to the ILM policy, you must ensure that the policy has at least one rule that filters by Object Size. Due to the overhead of managing the number of fragments associated with an erasure-coded copy, do not erasure code objects smaller than 200 KB.
  10. Optionally, add different time periods or create additional copies at different locations:
    • Click Add to add a different time period to the placement instructions.
    • Click the plus icon to create additional copies at a different location during the same time period.
  11. Click Refresh to update the Retention Diagram and to confirm your placement instructions.
    Each line in the diagram shows where and when object copies will be placed. The type of copy is represented by one of the following icons:
    • icon for replicated copiesReplicated copy
    • Icon for erasure coded copy Erasure-coded copy
    • Cloud Storage Pool icon Cloud Storage Pool copy
    In this example, three replicated copies will be saved to three storage pools (DC1, DC2, and DC3) for one year. Then, an erasure-coded copy will be saved for 10 years, using a 6+3 erasure-coding scheme. After 10 years, the objects will be deleted from StorageGRID.


    Retention diagram

  12. Click Save.
    The ILM rule is saved. The rule does not become active until it is added to an ILM policy and that policy is activated.