Skip to main content

Create an ONTAP S3 bucket lifecycle management rule

Contributors netapp-lenida netapp-mdavidson netapp-dbagwell netapp-manini netapp-aherbin
PDFs

Beginning with ONTAP 9.13.1, you can create lifecycle management rules to manage object lifecycles in your S3 buckets. You can define deletion rules for specific objects in a bucket, and through these rules, expire those bucket objects. This enables you to meet retention requirements and manage overall S3 object storage efficiently.

Note If object locking is enabled for your bucket objects, the lifecycle management rules for object expiration will not be applied on locked objects. For information about object locking, see Create a bucket.
Before you begin

  • An S3-enabled SVM containing an S3 server and a bucket must already exist. See Create an SVM for S3 for more information.

  • You should be aware that bucket lifecycle management rules are not supported in MetroCluster configurations.

About this task

When creating your lifecycle management rules, you can apply the following deletion actions to your bucket objects:

  • Deletion of current versions - This action expires objects identified by the rule. If versioning is enabled on the bucket, S3 makes all expired objects unavailable. If versioning is not enabled, this rule deletes the objects permanently. The CLI action is Expiration.

  • Deletion of non-current versions - This action specifies when S3 can permanently remove non-current objects. The CLI action is NoncurrentVersionExpiration.

    Note A non-current version is based on the current version's creation or modification time. Delayed removal of non-current objects can be helpful when you accidentally delete or overwrite an object. For example, you can configure an expiration rule to delete non-current versions five days after they become non-current. For example, suppose that on 1/1/2014 at 10:30 AM UTC, you create an object called photo.gif (version ID 111111). On 1/2/2014 at 11:30 AM UTC, you accidentally delete photo.gif (version ID 111111), which creates a delete marker with a new version ID (such as version ID 4857693). You now have five days to recover the original version of photo.gif (version ID 111111) before the deletion is permanent. On 1/8/2014 at 00:00 UTC, the Lifecycle rule for expiration runs and permanently deletes photo.gif (version ID 111111), five days after it became a non-current version.

  • Deletion of expired delete markers - This action deletes expired object delete markers. In versioning-enabled buckets, objects with a delete markers become the current versions of the objects. The objects are not deleted, and no action can be performed on them. These objects become expired when there are no current versions associated with them. The CLI action is Expiration.

  • Deletion of incomplete multipart uploads - This action sets a maximum time (in days) that you want to allow multipart uploads to remain in progress. Following which, they are deleted. The CLI action is AbortIncompleteMultipartUpload.

The procedure you follow depends on the interface that you use. With ONTAP 9.13,1, you need to use the CLI. Beginning with ONTAP 9.14.1, you can also use System Manager.

Manage lifecycle management rules with the CLI

Beginning with ONTAP 9.13.1, you can use the ONTAP CLI to create lifecycle management rules to expire objects in your S3 buckets.

Before you begin

For the CLI, you need to define the required fields for each expiration action type when creating a bucket lifecycle management rule. These fields can be modified after initial creation. The following table displays the unique fields for each action type.

Action type

Unique fields

NonCurrentVersionExpiration

  • -non-curr-days - Number of days after which non-current versions will be deleted

  • -new-non-curr-versions - Number of latest non-current versions to be retained

Expiration

  • -obj-age-days - Number of days since creation, after which current version of objects can be deleted

  • -obj-exp-date - Specific date when the objects should expire

  • -expired-obj-del-markers - Cleanup object delete markers

AbortIncompleteMultipartUpload

  • -after-initiation-days - Number of days of initiation, after which upload can be aborted

In order for the bucket lifecycle management rule to only be applied to a specific subset of objects, admins must set each filter when creating the rule. If these filters are not set when creating the rule, the rule will be applied to all objects within the bucket.

All filters can be modified after initial creation except for the following:

  • -prefix

  • -tags

  • -obj-size-greater-than

  • -obj-size-less-than

Steps

  1. Use the vserver object-store-server bucket lifecycle-management-rule create command with required fields for your expiration action type to create your bucket lifecycle management rule.

Example

The following command creates a NonCurrentVersionExpiration bucket lifecycle management rule:

vserver object-store-server bucket lifecycle-management-rule create -vserver <svm_name> -bucket <bucket_name> -rule-id <rule_name> -action NonCurrentVersionExpiration -index <lifecycle_rule_index_integer> -is-enabled {true|false} -prefix <object_name> -tags <text> -obj-size-greater-than {<integer>[KB|MB|GB|TB|PB]} -obj-size-less-than {<integer>[KB|MB|GB|TB|PB]} -new-non-curr-versions <integer> -non-curr-days <integer>
Example

The following command creates an Expiration bucket lifecycle management rule:

vserver object-store-server bucket lifecycle-management-rule create -vserver <svm_name> -bucket <bucket_name> -rule-id <rule_name> -action Expiration -index <lifecycle_rule_index_integer> -is-enabled {true|false} -prefix <object_name> -tags <text> -obj-size-greater-than {<integer>[KB|MB|GB|TB|PB]} -obj-size-less-than {<integer>[KB|MB|GB|TB|PB]} -obj-age-days <integer> -obj-exp-date <"MM/DD/YYYY HH:MM:SS"> -expired-obj-del-marker {true|false}
Example

The following command creates an AbortIncompleteMultipartUpload bucket lifecycle management rule:

vserver object-store-server bucket lifecycle-management-rule create -vserver <svm_name> -bucket <bucket_name> -rule-id <rule_name> -action AbortIncompleteMultipartUpload -index <lifecycle_rule_index_integer> -is-enabled {true|false} -prefix <object_name> -tags <text> -obj-size-greater-than {<integer>[KB|MB|GB|TB|PB]} -obj-size-less-than {<integer>[KB|MB|GB|TB|PB]} -after-initiation-days <integer>

Manage lifecycle management rules with System Manager

Beginning with ONTAP 9.14.1, you can expire S3 objects by using System Manager. You can add, edit, and delete lifecycle management rules for your S3 objects. Additionally, you can import a lifecycle rule created for one bucket and utilize it for the objects in another bucket. You can disable an active rule and enable it later.

Add a lifecycle management rule

  1. Click Storage > Buckets.

  2. Select the bucket for which you want to specify the expiration rule.

  3. Click the Menu options icon icon and select Manage lifecycle rules.

  4. Click Add > Lifecycle rule.

  5. On the Add a lifecycle rule page, add the name of the rule.

  6. Define the scope of the rule, whether you want it to apply to all the objects in the bucket or on specific objects. If you want to specify objects, add at least one of the following filter criteria:

    1. Prefix: Specify a prefix of the object key names to which the rule should apply. Typically, it is the path or folder of the object. You can enter one prefix per rule. Unless a valid prefix is provided, the rule applies to all the objects in a bucket.

    2. Tags: Specify up to three key and value pairs (tags) for the objects to which the rule should apply. Only valid keys are used for filtering. The value is optional. However, if you add values, ensure that you add only valid values for the corresponding keys.

    3. Size: You can limit the scope between the minimum and maximum sizes of the objects. You can enter either or both the values. The default unit is MiB.

  7. Specify the action:

    1. Expire the current version of objects: Set a rule to make all current objects permanently unavailable after a specific number of days since their creation, or on a specific date. This option is unavailable if the Delete expired object delete markers option is selected.

    2. Permanently delete non-current versions: Specify the number of days after which the non-current version is deleted, and the number of versions to retain.

    3. Delete expired object delete markers: Select this action to delete objects with expired delete markers, that is delete markers without an associated current object.

      Note This option becomes unavailable when you select the Expire the current version of objects option that automatically deletes all objects after the retention period. This option also becomes unavailable when object tags are used for filtering.

    4. Delete incomplete multipart uploads: Set the number of days after which incomplete multipart uploads are to be deleted. If the multipart uploads that are in progress fail within the specified retention period, you can delete the incomplete multipart uploads. This option becomes unavailable when object tags are used for filtering.

    5. Click Save.

Import a lifecycle rule

  1. Click Storage > Buckets.

  2. Select the bucket for which you want to import the expiration rule.

  3. Click the Menu options icon icon and select Manage lifecycle rules.

  4. Click Add > Import a rule.

  5. Select the bucket from which you want to import the rule. The lifecycle management rules defined for the selected bucket appear.

  6. Select the rule that you want to import. You have the option to select one rule at a time, with the default selection being the first rule.

  7. Click Import.

Edit, delete, or disable a rule

You can only edit the lifecycle management actions associated with the rule. If the rule was filtered with object tags, then the Delete expired object delete markers and Delete incomplete multipart uploads options are unavailable.

When you delete a rule, that rule will no longer apply to previously associated objects.

  1. Click Storage > Buckets.

  2. Select the bucket for which you want to edit, delete, or disable the lifecycle management rule.

  3. Click the Menu options icon icon and select Manage lifecycle rules.

  4. Select the required rule. You can edit and disable one rule at a time. You can delete multiple rules at once.

  5. Select Edit, Delete, or Disable, and complete the procedure.