How Cloud Tiering works Edit on GitHub Request doc changes

Contributors netapp-bcammett

Cloud Tiering is a NetApp-managed service that uses FabricPool technology to automatically tier inactive (cold) data from your on-premises ONTAP clusters to object storage in the cloud. Connections to ONTAP take place from the NetApp Service Connector.

At this time, Cloud Tiering can tier your inactive data to AWS S3 or Azure Blob storage. Support for additional object storage providers will be added later.

Overview

The following image shows the relationship between each component:

An architecture image that shows the Cloud Tiering service with a connection to the Service Connector in your cloud provider

At a high level, Cloud Tiering works like this:

  1. You deploy a NetApp Service Connector.

    • If you want to tier cold data to S3, the Service Connector can be in an AWS VPC or on your premises.

    • If you want to tier cold data to Blob storage, the Service Connector must be in an Azure VNet.

  2. The Service Connector communicates with ONTAP clusters to discover the amount of active and inactive data on the cluster.

  3. You choose the volumes to tier and the tiering policy to apply to those volumes.

  4. You choose the object storage provider that you want to use.

  5. The Service Connector configures ONTAP to use that object storage provider.

  6. The Service Connector configures data tiering on each volume that you selected.

  7. ONTAP starts tiering inactive data to the object store, as soon as the data has reached the thresholds to be considered inactive (see Volume tiering policies).

NetApp Service Connector

The Service Connector is software that communicates with ONTAP clusters to discover the amount of active and inactive data on the cluster and to set up data tiering. Cloud Tiering prompts you to deploy the Service Connector when you discover your first ONTAP cluster. Connections to ONTAP take place from the Service Connector. A single Service Connector can discover multiple ONTAP clusters.

Object storage

Each ONTAP cluster tiers inactive data to a single object store that Cloud Tiering creates for you. Cloud Tiering prompts you to choose a cloud storage provider when you set up tiering for each cluster.

S3 storage classes

When you set up data tiering to AWS, you can choose the S3 Standard storage class or the Standard-Infrequent Access storage class. If you choose Standard-Infrequent, inactive data starts in the Standard storage class and transitions to Standard-Infrequent Access, if the data is not accessed after 30 days.

Azure Blob access tiers

When you set up data tiering to Azure, Cloud Tiering automatically uses the Hot access tier for your inactive data. We plan to add support for the Cool access tier in a future release.

Volume tiering policies

When you select the volumes that you want to tier, you also choose the volume tiering policy to apply to those volumes. A tiering policy defines the type of cold data that you want to tier from the volumes.

Auto

ONTAP tiers all cold blocks in the volume to object storage. The cold data includes not just Snapshot copies but also cold user data from the active file system.

If read by random reads, cold data blocks on the cloud tier become hot and are moved to the performance tier. If read by sequential reads, such as those associated with index and antivirus scans, cold data blocks on the cloud tier stay cold and are not written to the performance tier.

Data is tiered only after an aggregate has reached 50% capacity and when the data has reached the cooling period, which is approximately 31 days. The cooling period is the time that user data in a volume must remain inactive for the data to be considered "cold" and moved to the object store.

Snapshot only

ONTAP tiers cold Snapshot blocks in the volume that are not shared with the active file system to object storage. If read, cold data blocks on the cloud tier become hot and are moved to the performance tier.

Data is tiered only after an aggregate has reached 50% capacity and when the data has reached the cooling period, which is approximately 2 days.