Backing up Cloud Volumes ONTAP data to Amazon S3

Contributors netapp-tonacki

Complete a few steps to get started backing up data from Cloud Volumes ONTAP to Amazon S3.

Quick start

Get started quickly by following these steps or scroll down to the remaining sections for full details.

One Verify support for your configuration

  • You’re running Cloud Volumes ONTAP 9.6 or later in AWS.

  • You have a valid cloud provider subscription for the storage space where your backups will be located.

  • You have subscribed to the Cloud Manager Marketplace Backup offering, an AWS annual contract, or you have purchased and activated a Cloud Backup BYOL license from NetApp.

  • The IAM role that provides the Cloud Manager Connector with permissions includes S3 permissions from the latest Cloud Manager policy.

Two Enable Cloud Backup on your new or existing system

  • New systems: Cloud Backup is enabled by default in the working environment wizard. Be sure to keep the option enabled.

  • Existing systems: Select the working environment and click Enable next to the Backup & Restore service in the right-panel, and then follow the setup wizard.

    A screenshot that shows the Backup & Restore Enable button which is available after you select a working environment.

Three Enter the provider details

Select the AWS Account and the region where you want to create the backups. You can also choose your own customer-managed key for data encryption instead of using the default Amazon S3 encryption key.

A screenshot that shows the cloud provider details when backing up volumes from a Cloud Volumes ONTAP system to AWS S3.

Four Define the default backup policy

The default policy backs up volumes every day and retains the most recent 30 backup copies of each volume. Change to hourly, daily, weekly, or monthly backups, or select one of the system-defined policies that provide more options. You can also change the number of backup copies you want to retain.

Backups are stored in S3 Standard storage by default. If your cluster is using ONTAP 9.10.1 or greater, you can choose to tier backups to either S3 Glacier or S3 Glacier Deep Archive storage after a certain number of days for further cost optimization.

A screenshot that shows the Cloud Backup settings where you can choose your backup schedule and retention period.

Five Select the volumes that you want to back up

Identify which volumes you want to back up using the default backup policy in the Select Volumes page. If you want to assign different backup policies to certain volumes, you can create additional policies and apply them to volumes later.

Requirements

Read the following requirements to make sure that you have a supported configuration before you start backing up volumes to S3.

The following image shows each component and the connections that you need to prepare between them:

A diagram showing how Cloud Backup communicates with the volumes on the source systems and the destination storage where the backup files are located.

When the Cloud Restore instance is deployed in the cloud, it is located in the same subnet as the Connector.

Supported ONTAP versions

Cloud Volumes ONTAP 9.6 and later.

License requirements

For Cloud Backup PAYGO licensing, a Cloud Manager subscription is available in the AWS Marketplace that enables deployments of Cloud Volumes ONTAP and Cloud Backup. You need to subscribe to this Cloud Manager subscription before you enable Cloud Backup. Billing for Cloud Backup is done through this subscription.

For an annual contract that enables you to back up both Cloud Volumes ONTAP data and on-premises ONTAP data, you need to subscribe from the AWS Marketplace page and then associate the subscription with your AWS credentials.

For an annual contract that enables you to bundle Cloud Volumes ONTAP and Cloud Backup, you must set up the annual contract when you create a Cloud Volumes ONTAP working environment. This option doesn’t enable you to back up on-prem data.

For Cloud Backup BYOL licensing, you need the serial number from NetApp that enables you to use the service for the duration and capacity of the license. Learn how to manage your BYOL licenses.

And you need to have an AWS account for the storage space where your backups will be located.

Supported AWS regions

Cloud Backup is supported in all AWS regions where Cloud Volumes ONTAP is supported; including AWS GovCloud regions.

Required setup for creating backups in a different AWS account

By default, backups are created using the same account as the one used for your Cloud Volumes ONTAP system. If you want to use a different AWS account for your backups, you must log in to the AWS portal and link the two accounts.

Required information for using customer-managed keys for data encryption

You can choose your own customer-managed keys for data encryption in the activation wizard instead of using the default Amazon S3 encryption keys. In this case you’ll need to have the encryption managed keys already set up. See how to use your own keys.

AWS Backup permissions required

The IAM role that provides Cloud Manager with permissions must include S3 permissions from the latest Cloud Manager policy.

Here are the specific permissions from the policy:

{
            "Sid": "backupPolicy",
            "Effect": "Allow",
            "Action": [
                "s3:DeleteBucket",
                "s3:GetLifecycleConfiguration",
                "s3:PutLifecycleConfiguration",
                "s3:PutBucketTagging",
                "s3:ListBucketVersions",
                "s3:GetObject",
                "s3:DeleteObject",
                "s3:PutObject",
                "s3:ListBucket",
                "s3:ListAllMyBuckets",
                "s3:GetBucketTagging",
                "s3:GetBucketLocation",
                "s3:GetBucketPolicyStatus",
                "s3:GetBucketPublicAccessBlock",
                "s3:GetBucketAcl",
                "s3:GetBucketPolicy",
                "s3:PutBucketPublicAccessBlock",
                "s3:PutEncryptionConfiguration",
                "athena:StartQueryExecution",
                "athena:GetQueryResults",
                "athena:GetQueryExecution",
                "glue:GetDatabase",
                "glue:GetTable",
                "glue:CreateTable",
                "glue:CreateDatabase",
                "glue:GetPartitions",
                "glue:BatchCreatePartition",
                "glue:BatchDeletePartition"
            ],
            "Resource": [
                "arn:aws:s3:::netapp-backup-*"
            ]
        },

If you deployed the Connector using version 3.9.15 or greater, these permissions should be part of the IAM role already. Otherwise you’ll need to add the missing permissions. Specifically the "athena" and "glue" permissions, as they are required for Search & Restore.

AWS Restore permissions required

The following EC2 permissions are needed for the IAM role that provides Cloud Manager with permissions so that it can start, stop, and terminate the Cloud Restore instance for Browse & Restore operations:

          "Action": [
              "ec2:DescribeInstanceTypeOfferings",
              "ec2:StartInstances",
              "ec2:StopInstances",
              "ec2:TerminateInstances"
          ],
Required outbound internet access for AWS deployments

The Cloud Restore instance requires outbound internet access. If your virtual or physical network uses a proxy server for internet access, ensure that the instance has outbound internet access to contact the following endpoints.

Endpoints Purpose

http://amazonlinux.us-east-1.amazonaws.com/2/extras/docker/stable/ x86_64/4bf88ee77c395ffe1e0c3ca68530dfb3a683ec65a4a1ce9c0ff394be50e922b2/

CentOS package for the Cloud Restore Instance AMI.

https://download.docker.com/linux/centos/docker-ce.repo

Provides the Docker Engine packages.

http://cloudmanagerinfraprod.azurecr.io
https://cloudmanagerinfraprod.azurecr.io

Cloud Restore Instance image repository.

Enabling Cloud Backup on a new system

Cloud Backup is enabled by default in the working environment wizard. Be sure to keep the option enabled.

See Launching Cloud Volumes ONTAP in AWS for requirements and details for creating your Cloud Volumes ONTAP system.

Steps

  1. Click Create Cloud Volumes ONTAP.

  2. Select Amazon Web Services as the cloud provider and then choose a single node or HA system.

  3. Fill out the Details & Credentials page.

  4. On the Services page, leave the service enabled and click Continue.

    Shows the Cloud Backup option in the working environment wizard.

  5. Complete the pages in the wizard to deploy the system.

Result

Cloud Backup is enabled on the system and backs up volumes every day and retains the most recent 30 backup copies.

What’s next?

You can start and stop backups for volumes or change the backup schedule.
You can also restore entire volumes or individual files from a backup file to a Cloud Volumes ONTAP system in AWS, or to an on-premises ONTAP system.

Enabling Cloud Backup on an existing system

Enable Cloud Backup at any time directly from the working environment.

Steps

  1. Select the working environment and click Enable next to the Backup & Restore service in the right-panel.

    A screenshot that shows the Cloud Backup Settings button which is available after you select a working environment.

  2. Select the provider details and click Next.

    1. The AWS Account used to store the backups. This can be a different account than where the Cloud Volumes ONTAP system resides.

      If you want to use a different AWS account for your backups, you must log in to the AWS portal and link the two accounts.

    2. The region where the backups will be stored. This can be a different region than where the Cloud Volumes ONTAP system resides.

    3. Whether you’ll use the default Amazon S3 encryption keys or choose your own customer-managed keys from your AWS account to manage encryption of your data. (See how to use your own encryption keys).

      A screenshot that shows the cloud provider details when backing up volumes from a Cloud Volumes ONTAP system to AWS S3.

  3. Enter the default backup policy details and click Next.

    1. Define the backup schedule and choose the number of backups to retain. See the list of existing policies you can choose.

    2. When using ONTAP 9.10.1 and greater, you can choose to tier backups to either S3 Glacier or S3 Glacier Deep Archive storage after a certain number of days for further cost optimization. Learn more about using archival tiers.

      A screenshot that shows the Cloud Backup settings where you can choose your schedule and backup retention.

  4. Select the volumes that you want to back up using the default backup policy in the Select Volumes page. If you want to assign different backup policies to certain volumes, you can create additional policies and apply them to those volumes later.

    A screenshot of selecting the volumes that will be backed up.

    • To back up all volumes, check the box in the title row (button backup all volumes).

    • To back up individual volumes, check the box for each volume (button backup 1 volume).

  5. If you want all volumes added in the future to have backup enabled, just leave the checkbox for "Automatically back up future volumes…​" checked. If you disable this setting, you’ll need to manually enable backups for future volumes.

  6. Click Activate Backup and Cloud Backup starts taking the initial backups of each selected volume.

Result

Cloud Backup starts taking the initial backups of each selected volume and the Volume Backup Dashboard is displayed so you can monitor the state of the backups.

What’s next?

You can start and stop backups for volumes or change the backup schedule.
You can also restore entire volumes or individual files from a backup file to a Cloud Volumes ONTAP system in AWS, or to an on-premises ONTAP system.