How Cloud Tiering works Edit on GitHub Request doc changes

Contributors netapp-bcammett

Cloud Tiering is a NetApp-managed service that uses FabricPool technology to automatically tier inactive (cold) data from your on-premises ONTAP clusters to object storage in the cloud. Connections to ONTAP take place from the NetApp Service Connector.

Overview

The following image shows the relationship between each component:

An architecture image that shows the Cloud Tiering service with a connection to the Service Connector in your cloud provider

At a high level, Cloud Tiering works like this:

  1. You deploy a NetApp Service Connector.

    • When tiering to S3, the Service Connector can be in an AWS VPC or on your premises.

    • When tiering to Blob storage, the Service Connector must be in an Azure VNet.

    • When tiering to Google Cloud Storage, the Service Connector must reside in a Google Cloud Platform VPC.

  2. The Service Connector communicates with ONTAP clusters to discover the amount of active and inactive data on the cluster.

  3. You choose the volumes to tier and the tiering policy to apply to those volumes.

  4. You choose the object storage provider that you want to use.

  5. The Service Connector configures ONTAP to use that object storage provider.

  6. The Service Connector configures data tiering on each volume that you selected.

  7. ONTAP starts tiering inactive data to the object store, as soon as the data has reached the thresholds to be considered inactive (see Volume tiering policies).

NetApp Service Connector

The Service Connector is software that communicates with ONTAP clusters to discover the amount of active and inactive data on the cluster and to set up data tiering. Cloud Tiering prompts you to deploy the Service Connector when you discover your first ONTAP cluster. Connections to ONTAP take place from the Service Connector. A single Service Connector can discover multiple ONTAP clusters.

Object storage

Each ONTAP cluster tiers inactive data to a single object store that Cloud Tiering creates for you. Cloud Tiering prompts you to choose a cloud storage provider when you set up tiering for each cluster.

S3 storage classes

When you set up data tiering to AWS, you can choose the S3 Standard storage class or the Standard-Infrequent Access storage class. If you choose Standard-Infrequent, inactive data starts in the Standard storage class and transitions to Standard-Infrequent Access, if the data is not accessed after 30 days.

Azure Blob access tiers

When you set up data tiering to Azure, Cloud Tiering automatically uses the Hot access tier for your inactive data. We plan to add support for the Cool access tier in a future release.

Google Cloud storage classes

Cloud Tiering uses the Standard access tier for your inactive data.

Volume tiering policies

When you select the volumes that you want to tier, you also choose a volume tiering policy to apply to each volume. A tiering policy determines when or whether the user data blocks of a volume are moved to the cloud.

No tiering policy

Keeps the data on a volume in the performance tier, preventing it from being moved to the cloud.

Cold snapshots (Snapshot only)

ONTAP tiers cold Snapshot blocks in the volume that are not shared with the active file system to object storage. If read, cold data blocks on the cloud tier become hot and are moved to the performance tier.

Data is tiered only after an aggregate has reached 50% capacity and when the data has reached the cooling period, which is approximately 2 days.

Writes from the cloud tier to the performance tier are disabled if performance tier capacity is greater than 70%. If this occurs, blocks are accessed directly from the cloud tier.
Cold user data (Auto)

ONTAP tiers all cold blocks in the volume (not including metadata) to object storage. The cold data includes not just Snapshot copies but also cold user data from the active file system.

If read by random reads, cold data blocks on the cloud tier become hot and are moved to the performance tier. If read by sequential reads, such as those associated with index and antivirus scans, cold data blocks on the cloud tier stay cold and are not written to the performance tier.

Data is tiered only after an aggregate has reached 50% capacity and when the data has reached the cooling period, which is approximately 31 days. The cooling period is the time that user data in a volume must remain inactive for the data to be considered "cold" and moved to the object store.

Writes from the cloud tier to the performance tier are disabled if performance tier capacity is greater than 70%. If this occurs, blocks are accessed directly from the cloud tier.
All user data (All)

All data (not including metadata) is immediately moved to the cloud tier. There is no need to wait 48 hours for new blocks in a volume to become cold. Note that blocks located in the volume prior to the All policy being set require 48 hours to become cold.

If read, cold data blocks on the cloud tier stay cold and are not written back to the performance tier. This policy is available starting with ONTAP 9.6.

Take the following into consideration before you choose this tiering policy:

  • Tiering data immediately reduces storage efficiencies (inline only).

  • You should use this policy only if you are confident that cold data on the volume will not change.

  • Object storage is not transactional and will result in significant fragmentation if subjected to change.

  • Consider the impact of SnapMirror transfers before assigning the All tiering policy to source volumes in data protection relationships.

    Because data is tiered immediately, SnapMirror will read data from the cloud tier rather than the performance tier. This will result in slower SnapMirror operations—​possibly slowing other SnapMirror operations later in queue—even if they are using different tiering policies.

All DP user data (Backup)

All data on a data protection volume (not including metadata) is immediately moved to the cloud tier. If read, cold data blocks on the cloud tier stay cold and are not written back to the performance tier (starting with ONTAP 9.4).

This policy is available for ONTAP 9.5 or earlier. It was replaced with the All tiering policy starting with ONTAP 9.6.