Tiering data from on-premises ONTAP clusters to Amazon S3

08/29/2023 Contributors

Free space on your on-prem ONTAP clusters by tiering inactive data to Amazon S3.

Quick start

Get started quickly by following these steps. Details for each step are provided in the following sections in this topic.

Identify the configuration method you'll use

Choose whether you'll connect your on-premises ONTAP cluster directly to AWS S3 over the public internet, or whether you'll use a VPN or AWS Direct Connect and route traffic through a private VPC Endpoint interface to AWS S3.

See the available connection methods.

Prepare your BlueXP Connector

If you already have a Connector deployed in your AWS VPC or on your premises, then you're all set. If not, then you'll need to create a Connector to tier ONTAP data to AWS S3 storage. You'll also need to customize network settings for the Connector so that it can connect to AWS S3.

See how to create a Connector and how to define required network settings.

Prepare your on-premises ONTAP cluster

Discover your ONTAP cluster in BlueXP, verify that the cluster meets minimum requirements, and customize network settings so the cluster can connect to AWS S3.

See how to get your on-premises ONTAP cluster ready.

Prepare Amazon S3 as your tiering target

Set up permissions for the Connector to create and manage the S3 bucket. You'll also need to set up permissions for the on-premises ONTAP cluster so it can read and write data to the S3 bucket.

See how to set up permissions for the Connector and for your on-prem cluster.

Enable BlueXP tiering on the system

Select an on-prem working environment, click Enable for the Tiering service, and follow the prompts to tier data to Amazon S3.

See how to enable Tiering for your volumes.

Set up licensing

After your free trial ends, pay for BlueXP tiering through a pay-as-you-go subscription, an ONTAP BlueXP tiering BYOL license, or a combination of both:

To subscribe from the AWS Marketplace, go to the BlueXP Marketplace offering, click Subscribe, and then follow the prompts.
To pay using a BlueXP tiering BYOL license, contact us if you need to purchase one, and then add it to your account from the BlueXP digital wallet.

Network diagrams for connection options

There are two connection methods you can use when configuring tiering from on-premises ONTAP systems to AWS S3.

Public connection - Directly connect the ONTAP system to AWS S3 using a public S3 endpoint.
Private connection - Use a VPN or AWS Direct Connect and route traffic through a VPC Endpoint interface that uses a private IP address.

The following diagram shows the public connection method and the connections that you need to prepare between the components. You can use a Connector that you've installed on your premises, or a Connector that you've deployed in the AWS VPC.

A diagram showing how BlueXP tiering communicates over a public connection with the volumes on the cluster and the AWS S3 storage where the inactive data is located.

The following diagram shows the private connection method and the connections that you need to prepare between the components. You can use a Connector that you've installed on your premises, or a Connector that you've deployed in the AWS VPC.

A diagram showing how BlueXP tiering communicates over a private connection with the volumes on the cluster and the AWS S3 storage where the inactive data is located.

Communication between a Connector and S3 is for object storage setup only.

Prepare your Connector

The BlueXP Connector is the main software for BlueXP functionality. A Connector is required to tier your inactive ONTAP data.

Creating or switching Connectors

If you already have a Connector deployed in your AWS VPC or on your premises, then you're all set. If not, then you'll need to create a Connector in either of those locations to tier ONTAP data to AWS S3 storage. You can't use a Connector that's deployed in another cloud provider.

Connector networking requirements

Ensure that the network where the Connector is installed enables the following connections:
- An HTTPS connection over port 443 to the BlueXP tiering service and to your S3 object storage (see the list of endpoints)
- An HTTPS connection over port 443 to your ONTAP cluster management LIF
Ensure that the Connector has permissions to manage the S3 bucket
If you have a Direct Connect or VPN connection from your ONTAP cluster to the VPC, and you want communication between the Connector and S3 to stay in your AWS internal network (a private connection), you'll need to enable a VPC Endpoint interface to S3. See how to set up a VPC endpoint interface.

Prepare your ONTAP cluster

Your ONTAP clusters must meet the following requirements when tiering data to Amazon S3.

ONTAP requirements

Supported ONTAP platforms

When using ONTAP 9.8 and later: You can tier data from AFF systems, or FAS systems with all-SSD aggregates or all-HDD aggregates.
When using ONTAP 9.7 and earlier: You can tier data from AFF systems, or FAS systems with all-SSD aggregates.

Supported ONTAP versions

ONTAP 9.2 or later
ONTAP 9.7 or later is required if you plan to use an AWS PrivateLink connection to object storage

Supported volumes and aggregates

The total number of volumes that BlueXP tiering can tier might be less than the number of volumes on your ONTAP system. That's because volumes can't be tiered from some aggregates. Refer to the ONTAP documentation for functionality or features not supported by FabricPool.

BlueXP tiering supports FlexGroup volumes starting with ONTAP 9.5. Setup works the same as any other volume.

Cluster networking requirements

The cluster requires an inbound HTTPS connection from the Connector to the cluster management LIF.

A connection between the cluster and the BlueXP tiering service is not required.
An intercluster LIF is required on each ONTAP node that hosts the volumes you want to tier. These intercluster LIFs must be able to access the object store.

The cluster initiates an outbound HTTPS connection over port 443 from the intercluster LIFs to Amazon S3 storage for tiering operations. ONTAP reads and writes data to and from object storage — the object storage never initiates, it just responds.
The intercluster LIFs must be associated with the IPspace that ONTAP should use to connect to object storage. Learn more about IPspaces.

When you set up BlueXP tiering, you are prompted for the IPspace to use. You should choose the IPspace that these LIFs are associated with. That might be the "Default" IPspace or a custom IPspace that you created.

If you use are using a different IPspace than "Default", then you might need to create a static route to get access to the object storage.

All intercluster LIFs within the IPspace must have access to the object store. If you can't configure this for the current IPspace, then you'll need to create a dedicated IPspace where all intercluster LIFs have access to the object store.
If you are using a Private VPC Interface Endpoint in AWS for the S3 connection, then in order for HTTPS/443 to be used, you'll need to load the S3 endpoint certificate into the ONTAP cluster. See how to set up a VPC endpoint interface and load the S3 certificate.
Ensure that your ONTAP cluster has permissions to access the S3 bucket.

Discover your ONTAP cluster in BlueXP

You need to discover your on-premises ONTAP cluster in BlueXP before you can start tiering cold data to object storage. You'll need to know the cluster management IP address and the password for the admin user account to add the cluster.

Learn how to discover a cluster.

Prepare your AWS environment

When you set up data tiering for a new cluster, you're prompted whether you want the service to create an S3 bucket or if you want to select an existing S3 bucket in the AWS account where the Connector is set up. The AWS account must have permissions and an access key that you can enter in BlueXP tiering. The ONTAP cluster uses the access key to tier data in and out of S3.

By default, the tiering service creates the bucket for you. If you want to use your own bucket, you can create one before you start the tiering activation wizard and then select that bucket in the wizard. See how to create S3 buckets from BlueXP. The bucket must be used exclusively for storing inactive data from your volumes - it cannot be used for any other purpose. The S3 bucket must be in a region that supports BlueXP tiering.

If you are planning to configure BlueXP tiering to use a lower cost storage class where your tiered data will transition to after a certain number of days, you must not select any lifecycle rules when setting up the bucket in your AWS account. BlueXP tiering manages the lifecycle transitions.

Set up S3 permissions

You'll need to configure two sets of permissions:

Permissions for the Connector so it can create and manage the S3 bucket.
Permissions for the on-premises ONTAP cluster so it can read and write data to the S3 bucket.

Steps

Connector permissions:
- Confirm that these S3 permissions are part of the IAM role that provides the Connector with permissions. They should have been included by default when you first deployed the Connector. If not, you'll need to add any missing permissions. See the AWS Documentation: Editing IAM policies for instructions.
- The default bucket that BlueXP tiering creates has a prefix of "fabric-pool". If you want to use a different prefix for your bucket, you'll need to customize the permissions with the name you want to use. In the S3 permissions you'll see a line "Resource": ["arn:aws:s3:::fabric-pool*"]. You'll need to change "fabric-pool" to the prefix that you want to use. For example, if you want to use "tiering-1" as the prefix for your buckets, you'll change this line to "Resource": ["arn:aws:s3:::tiering-1*"].
  
  If you want to use a different prefix for buckets that you'll use for additional clusters in this same BlueXP account, you can add another line with the prefix for other buckets. For example:
  
  "Resource": ["arn:aws:s3:::tiering-1*"]
  "Resource": ["arn:aws:s3:::tiering-2*"]
  
  If you are creating your own bucket and do not use a standard prefix, you should change this line to "Resource": ["arn:aws:s3:::*"] so that any bucket is recognized. However, this may expose all your buckets instead of those you have designed to hold inactive data from your volumes.
Cluster permissions:
- When activating the service, the Tiering wizard will prompt you to enter an access key and secret key. These credentials are passed to the ONTAP cluster so that ONTAP can tier data to the S3 bucket. For that, you'll need to create an IAM user with the following permissions:
  "s3:ListAllMyBuckets", "s3:ListBucket", "s3:GetBucketLocation", "s3:GetObject", "s3:PutObject", "s3:DeleteObject"
  See the AWS Documentation: Creating a Role to Delegate Permissions to an IAM User for details.
Create or locate the access key.

BlueXP tiering passes the access key on to the ONTAP cluster. The credentials are not stored in the BlueXP tiering service.

AWS Documentation: Managing Access Keys for IAM Users

Configure your system for a private connection using a VPC endpoint interface

If you plan to use a standard public internet connection, then all the permissions are set by the Connector and there is nothing else you need to do. This type of connection is shown in the first diagram above.

If you want to have a more secure connection over the internet from your on-prem data center to the VPC, there's an option to select an AWS PrivateLink connection in the Tiering activation wizard. It's required if you plan to use a VPN or AWS Direct Connect to connect your on-premises system through a VPC Endpoint interface that uses a private IP address. This type of connection is shown in the second diagram above.

Create an Interface endpoint configuration using the Amazon VPC console or the command line. See details about using AWS PrivateLink for Amazon S3.
Modify the security group configuration that's associated with the BlueXP Connector. You must change the policy to "Custom" (from "Full Access"), and you must add the required S3 Connector permissions as shown earlier.

If you're using port 80 (HTTP) for communication to the private endpoint, you're all set. You can enable BlueXP tiering on the cluster now.

If you're using port 443 (HTTPS) for communication to the private endpoint, you must copy the certificate from the VPC S3 endpoint and add it to your ONTAP cluster, as shown in the next 4 steps.
Obtain the DNS name of the endpoint from the AWS Console.
Obtain the certificate from the VPC S3 endpoint. You do this by logging into the VM that hosts the BlueXP Connector and running the following command. When entering the DNS name of the endpoint, add “bucket” to the beginning, replacing the “*”:
```
[ec2-user@ip-10-160-4-68 ~]$ openssl s_client -connect bucket.vpce-0ff5c15df7e00fbab-yxs7lt8v.s3.us-west-2.vpce.amazonaws.com:443 -showcerts
```

From the output of this command, copy the data for the S3 certificate (all data between, and including, the BEGIN / END CERTIFICATE tags):

Certificate chain
0 s:/CN=s3.us-west-2.amazonaws.com`
   i:/C=US/O=Amazon/OU=Server CA 1B/CN=Amazon
-----BEGIN CERTIFICATE-----
MIIM6zCCC9OgAwIBAgIQA7MGJ4FaDBR8uL0KR3oltTANBgkqhkiG9w0BAQsFADBG
…
…
GqvbOz/oO2NWLLFCqI+xmkLcMiPrZy+/6Af+HH2mLCM4EsI2b+IpBmPkriWnnxo=
-----END CERTIFICATE-----

Log into the ONTAP cluster CLI and apply the certificate you copied using the following command (substitute your own storage VM name):
```
cluster1::> security certificate install -vserver <svm_name> -type server-ca
Please enter Certificate: Press <Enter> when done
```

Tier inactive data from your first cluster to Amazon S3

After you prepare your AWS environment, start tiering inactive data from your first cluster.

What you'll need

An on-premises working environment.
An AWS access key for an IAM user who has the required S3 permissions.

Steps

Select the on-prem ONTAP working environment.
Click Enable for the Tiering service from the right panel.

If the Amazon S3 tiering destination exists as a working environment on the Canvas, you can drag the cluster onto the working environment to initiate the setup wizard.
Define Object Storage Name: Enter a name for this object storage. It must be unique from any other object storage you may be using with aggregates on this cluster.
Select Provider: Select Amazon Web Services and click Continue.
Complete the sections in the Tiering Setup page:
1. S3 Bucket: Add a new S3 bucket or select an existing S3 bucket, select the bucket region, and click Continue.
  
  When using an on-prem Connector, you must enter the AWS Account ID that provides access to the existing S3 bucket or new S3 bucket that will be created.
  
  The fabric-pool prefix is used by default because the IAM policy for the Connector enables the instance to perform S3 actions on buckets named with that exact prefix. For example, you could name the S3 bucket fabric-pool-AFF1, where AFF1 is the name of the cluster. You can define the prefix for the buckets used for tiering as well. See setting up S3 permissions to make sure you have AWS permissions that recognize any custom prefix you plan to use.
2. Storage Class: BlueXP tiering manages the lifecycle transitions of your tiered data. Data starts in the Standard class, but you can create a rule to apply a different storage class to the data after a certain number of days.
  
  Select the S3 storage class that you want to transition the tiered data to and the number of days before the data is assigned to that class, and click Continue. For example, the screenshot below shows that tiered data is assigned to the Standard-IA class from the Standard class after 45 days in object storage.
  
  If you choose Keep data in this storage class, then the data remains in the Standard storage class and no rules are applied. See supported storage classes.
  
  Note that the lifecycle rule is applied to all objects in the selected bucket.
3. Credentials: Enter the access key ID and secret key for an IAM user who has the required S3 permissions, and click Continue.
  
  The IAM user must be in the same AWS account as the bucket that you selected or created on the S3 Bucket page.
4. Networking: Enter the networking details and click Continue.
  
  Select the IPspace in the ONTAP cluster where the volumes you want to tier reside. The intercluster LIFs for this IPspace must have outbound internet access so that they can connect to your cloud provider's object storage.
  
  Optionally, choose whether you'll use an AWS PrivateLink that you have previously configured. See the setup information above. A dialog box is displayed to help guide you through the endpoint configuration.
  
  You can also set the network bandwidth available to upload inactive data to object storage by defining the "Maximum transfer rate". Select the Limited radio button and enter the maximum bandwidth that can be used, or select Unlimited to indicate that there is no limit.
On the Tier Volumes page, select the volumes that you want to configure tiering for and launch the Tiering Policy page:
- To select all volumes, check the box in the title row () and click Configure volumes.
- To select multiple volumes, check the box for each volume () and click Configure volumes.
- To select a single volume, click the row (or icon) for the volume.
In the Tiering Policy dialog, select a tiering policy, optionally adjust the cooling days for the selected volumes, and click Apply.

Learn more about volume tiering policies and cooling days.

Result

You've successfully set up data tiering from volumes on the cluster to S3 object storage.

What's next?

Be sure to subscribe to the BlueXP tiering service.

You can review information about the active and inactive data on the cluster. Learn more about managing your tiering settings.

You can also create additional object storage in cases where you may want to tier data from certain aggregates on a cluster to different object stores. Or if you plan to use FabricPool Mirroring where your tiered data is replicated to an additional object store. Learn more about managing object stores.