Skip to main content

Create ILM policies

Contributors netapp-lhalbert
PDFs

Create one or more ILM policies to meet your quality-of-service requirements.

Having one active ILM policy allows you to apply the same ILM rules to all tenants and buckets.

Having multiple active ILM policies allows you to apply the appropriate ILM rules to specific tenants and buckets to fulfill multiple quality-of-service requirements.

Create an ILM policy

About this task

Before creating your own policy, verify that the default ILM policy does not meet your storage requirements.

Note Only use the system-provided policies, 2 copies Policy (for one-site grids) or 1 Copy per Site (for multi-site grids), in test systems. For StorageGRID 11.6 and earlier, the default rule in this policy uses the All Storage Nodes storage pool, which contains all sites. If your StorageGRID system has more than one site, two copies of an object might be placed on the same site.
Note If the global S3 Object Lock setting has been enabled, you must ensure that the ILM policy is compliant with the requirements of buckets that have S3 Object Lock enabled. In this section, follow the instructions that mention having S3 Object Lock enabled.
Before you begin

See also Use ILM policies.

Steps

  1. Select ILM > Policies.

    If the global S3 Object Lock setting is enabled, the ILM policies page indicates which ILM rules are compliant.

  2. Determine how you want to create the ILM policy.

    Create new policy

    1. Select Create policy.

    Clone existing policy

    1. Select the checkbox for the policy you want to start with, then select Clone.

    Edit existing policy

    1. If a policy is inactive, you can edit it. Select the checkbox for the inactive policy you want to start with, then select Edit.

  1. In the Policy name field, enter a unique name for the policy.

  2. Optionally, in the Reason for change field, enter the reason you are creating a new policy.

  3. To add rules to the policy, select Select rules. Select a rule name to view the settings for that rule.

    If you are cloning a policy:

    • The rules used by the policy you are cloning are selected.

    • If the policy you are cloning used any rules with no filters that were not the default rule, you are prompted to remove all but one of those rules.

    • If the default rule used a filter, you are prompted to select a new default rule.

    • If the default rule was not the last rule, you can move the rule to the end of the new policy.

    S3 Object Lock not enabled

    1. Select one default rule for the policy. To create a new default rule, select ILM rules page.

      The default rule applies to any objects that don't match another rule in the policy. The default rule can't use any filters and is always evaluated last.

      Note Don't use the Make 2 Copies rule as the default rule for a policy. The Make 2 Copies rule uses a single storage pool, All Storage Nodes, which contains all sites. If your StorageGRID system has more than one site, two copies of an object might be placed on the same site.
    S3 Object Lock enabled

    1. Select one default rule for the policy. To create a new default rule, select ILM rules page.

      The list of rules contains only the rules that are compliant and don't use any filters.

      Note Don't use the Make 2 Copies rule as the default rule for a policy. The Make 2 Copies rule uses a single storage pool, All Storage Nodes, which contains all sites. If you use this rule, multiple copies of an object might be placed on the same site.

    2. If you need a different "default" rule for objects in non-compliant S3 buckets, select Include a rule without filters for non-compliant S3 buckets, and select one non-compliant rule that does not use a filter.

      For example, you might want to use a Cloud Storage Pool to store objects in buckets that don't have S3 Object Lock enabled.

      Note You can only select one non-compliant rule that does not use a filter.

    See also Example 7: Compliant ILM policy for S3 Object Lock.

  1. When you are done selecting the default rule, select Continue.

  2. For the Other rules step, select any other rules you want to add to the policy. These rules use at least one filter (tenant account, bucket name, advanced filter, or the Noncurrent reference time). Then select Select.

    The Create a policy window now lists the rules you selected. The default rule is at the end, with the other rules above it.

    If S3 Object Lock is enabled and you also selected a non-compliant "default" rule, that rule is added as the second-to-last rule in the policy.

    Note A warning appears if any rule does not retain objects forever. When you activate this policy, you must confirm that you want StorageGRID to delete objects when the placement instructions for the default rule elapse (unless a bucket lifecycle keeps the objects for a longer time period).

  3. Drag the rows for the non-default rules to determine the order in which these rules will be evaluated.

    You can't move the default rule. If S3 Object Lock is enabled, you also can't move the non-compliant "default" rule if one was selected.

    Note You must confirm that the ILM rules are in the correct order. When the policy is activated, new and existing objects are evaluated by the rules in the order listed, starting at the top.

  4. As required, select Select rules to add or remove rules.

  5. When you are done, select Save.

  6. Repeat these steps to create additional ILM policies.

  7. Simulate an ILM policy. You should always simulate a policy before activating it to ensure it works as expected.

Simulate a policy

Simulate a policy on test objects before activating the policy and applying it to your production data.

Before you begin

  • You know the S3 bucket/object-key for each object you want to test.

Steps

  1. Using an S3 client or the S3 Console, ingest the objects required to test each rule.

  2. On the ILM policies page, select the checkbox for the policy, then select Simulate.

  3. In the Object field, enter the S3 bucket/object-key for a test object. For example, bucket-01/filename.png.

  4. If S3 versioning is enabled, optionally enter a version ID for the object in the Version ID field.

  5. Select Simulate.

  6. In the Simulation results section, confirm that each object was matched by the correct rule.

  7. To determine which storage pool or erasure-coding profile is in effect, select the name of the matched rule to go to the rule details page.

Caution Review any changes to the placement of existing replicated and erasure-coded objects. Changing an existing object's location might result in temporary resource issues when the new placements are evaluated and implemented.
Results

Any edits to the policy's rules will be reflected in the Simulation results and show the new match and previous match. The Simulate policy window retains the objects you tested until you select either Clear all or the remove icon Remove icon for each object in the Simulation results list.

Related information

Example ILM policy simulations

Activate a policy

When you activate a single new ILM policy, existing objects and newly ingested objects are managed by that policy. When you activate multiple policies, ILM policy tags assigned to buckets determine the objects to be managed.

Before you activate a new policy:

  1. Simulate the policy to confirm that it behaves as you expect.

  2. Review any changes to the placement of existing replicated and erasure-coded objects. Changing an existing object's location might result in temporary resource issues when the new placements are evaluated and implemented.

Caution Errors in an ILM policy can cause unrecoverable data loss.
About this task

When you activate an ILM policy, the system distributes the new policy to all nodes. However, the new active policy might not actually take effect until all grid nodes are available to receive the new policy. In some cases, the system waits to implement a new active policy to ensure that grid objects aren't accidentally removed. Specifically:

  • If you make policy changes that increase data redundancy or durability, those changes are implemented immediately. For example, if you activate a new policy that includes a three-copies rule instead of a two-copies rule, that policy will be implemented right away because it increases data redundancy.

  • If you make policy changes that could decrease data redundancy or durability, those changes will not be implemented until all grid nodes are available. For example, if you activate a new policy that uses a two-copies rule instead of a three-copies rule, the new policy will appear in the Active policy tab but it will not take effect until all nodes are online and available.

Steps

Follow the steps for activating one policy or multiple policies:

Activate one policy

Follow these steps if you will have only one active policy. If you already have one or more active policies and you are activating additional policies, follow the steps for activating multiple policies.

  1. When you are ready to activate a policy, select ILM > Policies.

    Alternatively, you can activate a single policy from the ILM > Policy tags page.

  2. On the Policies tab, select the checkbox for the policy you want to activate, then select Activate.

  3. Follow the appropriate step:

    • If a warning message prompts you to confirm that you want to activate the policy, select OK.

    • If a warning message containing details about the policy appears:

      1. Review the details to ensure the policy would manage data as expected.

      2. If the default rule stores objects for a limited number of days, review the retention diagram and then type in that number of days into the text box.

      3. If the default rule stores objects forever, but one or more other rules has limited retention, type yes in the text box.

      4. Select Activate policy.

Activate multiple policies

To activate multiple policies, you must create tags and assign a policy to each tag.

Tip When multiple tags are in use, if tenants frequently reassign policy tags to buckets, grid performance might be impacted. If you have untrusted tenants, consider using only the Default tag.

  1. Select ILM > Policy tags.

  2. Select Create.

  3. In the Create policy tag dialog box, type a tag name and, optionally, a description for the tag.

    Note Tag names and descriptions are visible to tenants. Choose values that will help tenants make an informed decision when selecting policy tags to assign to their buckets. For example, if the assigned policy will delete objects after a period of time, you could communicate that in the description. Do not include sensitive information in these fields.

  4. Select Create tag.

  5. In the ILM policy tags table, use the pull-down to select a policy to assign to the tag.

  6. If warnings appear in the Policy limitations column, select View policy details to review the policy.

  7. Ensure each policy would manage data as expected.

  8. Select Activate assigned policies. Or, select Clear changes to remove the policy assignment.

  9. In the Activate policies with new tags dialog box, review the descriptions of how each tag, policy, and rule will manage objects. Make changes as needed to ensure the policies will manage objects as expected.

  10. When you are sure you want to activate the policies, type yes in the text box, then select Activate policies.

Related information

Example 6: Changing an ILM policy