Skip to main content
Cloud Tiering

How Cloud Tiering works

Contributors netapp-bcammett
PDFs

Cloud Tiering is a NetApp-managed service that uses FabricPool technology to automatically tier inactive (cold) data from your on-premises ONTAP clusters to object storage in your public cloud or private cloud. Connections to ONTAP take place from the NetApp Service Connector.

Overview

The following image shows the relationship between each component:

An architecture image that shows the Cloud Tiering service with a connection to the Service Connector in your cloud provider, the Service Connector with a connection to your ONTAP cluster, and a connection between the ONTAP cluster and object storage in your cloud provider. Active data resides in the ONTAP cluster, while inactive data resides in object storage.

At a high level, Cloud Tiering works like this:

  1. You provide Cloud Tiering with details about your cluster and deploy a NetApp Service Connector.

    • When tiering to S3, the Service Connector can be in an AWS VPC or on your premises.

    • When tiering to Blob storage, the Service Connector must be in an Azure VNet.

    • When tiering to Google Cloud Storage, the Service Connector must reside in a Google Cloud Platform VPC.

    • When tiering to StorageGRID, the Service Connector must reside on your premises.

  2. You provide details about your object storage, including the bucket/container and a storage class or access tier.

  3. The Service Connector configures ONTAP to use the object storage provider and discovers the amount of active and inactive data on the cluster.

  4. You choose the volumes to tier and the tiering policy to apply to those volumes.

  5. ONTAP starts tiering inactive data to the object store, as soon as the data has reached the thresholds to be considered inactive (see Volume tiering policies).

NetApp Service Connector

The Service Connector is software that communicates with ONTAP clusters to discover the amount of active and inactive data on the cluster and to set up data tiering. Cloud Tiering prompts you to deploy the Service Connector when you discover your first ONTAP cluster. Connections to ONTAP take place from the Service Connector. A single Service Connector can discover multiple ONTAP clusters.

Object storage

Each ONTAP cluster tiers inactive data to a single object store. When you set up data tiering, you have the choice to add a new bucket/container or to select an existing bucket/container, along with a storage class or access tier when tiering to the public cloud.

Volume tiering policies

When you select the volumes that you want to tier, you also choose a volume tiering policy to apply to each volume. A tiering policy determines when or whether the user data blocks of a volume are moved to the cloud.

No tiering policy

Keeps the data on a volume in the performance tier, preventing it from being moved to the cloud.

Cold snapshots (Snapshot only)

ONTAP tiers cold Snapshot blocks in the volume that are not shared with the active file system to object storage. If read, cold data blocks on the cloud tier become hot and are moved to the performance tier.

Data is tiered only after an aggregate has reached 50% capacity and when the data has reached the cooling period. The default number of cooling days is 2, but you can adjust the number of days.

Note Writes from the cloud tier to the performance tier are disabled if performance tier capacity is greater than 70%. If this occurs, blocks are accessed directly from the cloud tier.
Cold user data (Auto)

ONTAP tiers all cold blocks in the volume (not including metadata) to object storage. The cold data includes not just Snapshot copies but also cold user data from the active file system.

If read by random reads, cold data blocks on the cloud tier become hot and are moved to the performance tier. If read by sequential reads, such as those associated with index and antivirus scans, cold data blocks on the cloud tier stay cold and are not written to the performance tier.

Data is tiered only after an aggregate has reached 50% capacity and when the data has reached the cooling period. The cooling period is the time that user data in a volume must remain inactive for the data to be considered "cold" and moved to the object store. The default number of cooling days is 31, but you can adjust the number of days.

Note Writes from the cloud tier to the performance tier are disabled if performance tier capacity is greater than 70%. If this occurs, blocks are accessed directly from the cloud tier.
All user data (All)

All data (not including metadata) is immediately marked as cold and tiered to object storage as soon as possible. There is no need to wait 48 hours for new blocks in a volume to become cold. Note that blocks located in the volume prior to the All policy being set require 48 hours to become cold.

If read, cold data blocks on the cloud tier stay cold and are not written back to the performance tier. This policy is available starting with ONTAP 9.6.

Take the following into consideration before you choose this tiering policy:

  • Tiering data immediately reduces storage efficiencies (inline only).

  • You should use this policy only if you are confident that cold data on the volume will not change.

  • Object storage is not transactional and will result in significant fragmentation if subjected to change.

  • Consider the impact of SnapMirror transfers before assigning the All tiering policy to source volumes in data protection relationships.

    Because data is tiered immediately, SnapMirror will read data from the cloud tier rather than the performance tier. This will result in slower SnapMirror operations—​possibly slowing other SnapMirror operations later in queue—even if they are using different tiering policies.

All DP user data (Backup)

All data on a data protection volume (not including metadata) is immediately moved to the cloud tier. If read, cold data blocks on the cloud tier stay cold and are not written back to the performance tier (starting with ONTAP 9.4).

Note This policy is available for ONTAP 9.5 or earlier. It was replaced with the All tiering policy starting with ONTAP 9.6.