Getting started with the Cloud Sync service Edit on GitHub Request doc changes

Getting started with the Cloud Sync service includes a few steps.

number 1 Verify requirements for the source and target

The most important requirement is verifying connectivity between the data broker and the source and target locations.

number 2 Choose where to run the NetApp data broker

The NetApp data broker software syncs data from a source to a target (this is called a sync relationship). You can run the data broker in AWS, Azure, Google Cloud Platform, or on your premises. Cloud Sync guides you through the installation process when you create a sync relationship.

number3 Create your first sync relationship

Start your free trial from Cloud Central, drag and drop your selections for the source and target, and follow the prompts to complete the setup.

number4 Pay for your sync relationships after your free trial ends

You can pay-as-you-go or purchase licenses directly from NetApp. Just go to the License Settings page in Cloud Sync.

Source and target requirements

The source and target in a supported sync relationship must meet the following requirements:

Networking requirements
  • The source and target must have a network connection to the data broker.

    For example, if an NFS server is in your data center and the data broker is in AWS, then you need a network connection (VPN or Direct Connect) from your network to the VPC.

  • NetApp recommends configuring the source, target, and data broker to use a Network Time Protocol (NTP) service. The time difference between the three components should not exceed 5 minutes.

Requirements for SMB/CIFS and NFS servers
  • The SMB/CIFS or NFS server can be a NetApp system or a non-NetApp system.

  • The file server must allow the data broker host to access the exports.

  • NFS versions 3, 4.1, and 4.2 are supported.

    The desired version must be enabled on the server.

  • SMB versions 1.0, 2.0, 2.1, and 3.0 are supported.

  • If you want Cloud Sync to copy ACLs between SMB/CIFS shares, then you must run a deployment script on a Windows host and activate support when you create a sync relationship.

  • If you want to sync NFS data from an ONTAP system, you must ensure that access to the NFS export list for an SVM is enabled (vserver nfs modify -vserver svm_name -showmount enabled).

    The default setting for showmount is enabled starting with ONTAP 9.2.
Requirements for Azure Blob storage

When creating a sync relationship between an Azure Blob container and an NFS or CIFS server, you need to provide Cloud Sync with the storage account connection string:

Shows a connection string

If you want to sync data between two Azure Blob containers, then the connection string must include a shared access signature (SAS). You also have the option to use a SAS when syncing between a Blob container and an NFS or CIFS server.

The SAS must allow access to the Blob service and all resource types (Service, Container, and Object). The SAS must also include the following permissions:

  • For the source Blob container: Read and List

  • For the target Blob container: Read, Write, List, Add, and Create

    Shows a shared access signature

Requirements for Google Cloud Storage

When setting up a relationship, you need to provide a project ID, client email, and private key for a Cloud Storage service account.

Data broker requirements

The data broker can reside in AWS, in Azure, in Google Cloud Platform, or in your network. You should be aware of the following requirements:

Supported regions
  • AWS: All regions are supported except for the China and GovCloud (US) regions.

  • Azure: All regions are supported except for the China, US Gov, and US DoD regions.

  • Google Cloud Platform: All regions are supported.

Networking requirements
  • The data broker needs an outbound internet connection so it can poll the Cloud Sync service for tasks.

    Cloud Sync configures outbound internet access by default when you install a data broker. Note that you can configure the data broker to use a proxy server during the installation process.

    If you need to limit outbound connectivity, see the list of endpoints that the data broker contacts.

  • Port 443 must be open on the data broker host for outbound traffic.

  • NetApp recommends configuring the source, target, and data broker to use a Network Time Protocol (NTP) service. The time difference between the three components should not exceed 5 minutes.

Permissions required to deploy the data broker in AWS

The AWS user account that you use to deploy the data broker must have the permissions included in the NetApp-provided policy.

Permissions required to deploy the data broker in Google Cloud Platform

Ensure that the following permissions are in place for your user account, as well as the service account that you use to create the data broker: Compute Admin, Deployment Manager Editor, and Service Account User.

Permissions required to prepare an existing Linux host for AWS access

When you install the data broker on an existing Linux host, the instructions included in the Cloud Sync interface prompt you to create an AWS user that has programmatic access and the permissions included in the NetApp-provided policy. Watch the installation video for more details.

Linux host requirements

If you install Cloud Manager on your own host, then you must verify support for your configuration.

  • Operating system: Red Hat Enterprise Linux 7 or CentOS 7 or Fedora

    The command yum update all must be run on the host before you install the data broker.

    A Red Hat Enterprise Linux system must be registered with Red Hat Subscription Management. If it is not registered, the system cannot access repositories to update required 3rd party software during installation.

  • RAM: 16 GB

  • CPU: 4 cores

  • Free disk space: 10 GB

Creating your first sync relationship

When you create a sync relationship, the Cloud Sync service copies files from the source to the target. After the initial copy, the service syncs any changed data every 24 hours.

Steps
  1. Go to NetApp Cloud Central.

  2. Sign up or log in and then start a free trial of Cloud Sync.

  3. After you log in, review details about using the service after the free trial ends, and then click OK.

  4. On the Select Source & Target page, choose a source and target.

    The following steps provide an example of how to create a sync relationship from an NFS server to an S3 bucket.

  5. Review the details about how the service works and then click Continue.

  6. On the NFS Server page, enter the IP address or fully qualified domain name of the NFS server that you want to sync to AWS.

  7. On the Data Broker page, follow the prompts to create a data broker virtual machine in AWS, Azure, or Google Cloud Platform, or to install the data broker software an existing Linux host.

    If you choose AWS, the AWS CloudFormation Console loads in a separate browser tab. The Cloud Sync service uses a CloudFormation template to quickly provision the data broker instance. The Cloud Sync service does not have access to your AWS credentials because you are logging in directly to AWS.

    AWS example

    The following video shows how to launch the data broker instance in AWS:

    The instance takes approximately 5 to 10 minutes to deploy. You can monitor the progress from the AWS CloudFormation console, or you can return to the Cloud Sync service website, which automatically refreshes when the instance is available.

    Existing host example

    The following video shows how to install the data broker software on an on-premises Linux host, or on an existing Linux host in the cloud:

    If you install the software using AWS keys, the keys are stored on the data broker. NetApp does not use the keys outside of the data broker.
  8. After you install the data broker, click Continue.

    The following image shows a successfully deployed instance in AWS:

    This screenshot shows a successfully deployed instance. The interface shows details about the instance including AWS networking.

  9. On the Directories page, optionally filter source objects and then select a top-level directory or subdirectory.

    Filtering source objects enables you to define how source files and folders are synced and maintained in the target location. For example, you can choose to never delete files from the target location, even if they were deleted from the source, which is useful for archiving.

    If you want to sync more than one directory on the NFS server, then you must create additional sync relationships after you are done.

    The following video shows how to filter source objects and select a directory:

  10. On the S3 Bucket page, choose the target location:

  11. On the Relationship Tags page, enter up to 9 relationship tags and then click Continue.

    The Cloud Sync service assigns the tags to each object that it syncs to the S3 bucket.

  12. Review the details of the sync relationship and then click Create Relationship.

  13. After the Cloud Sync service successfully creates the relationship, click View in Dashboard to view details about the data sync relationship.

Result

Congratulations, you have created your first sync relationship.

Paying for sync relationships after your free trial ends

There are two ways to pay for sync relationships after your 14-day free trial ends. The first option is to pay-as-you-go. The second option is to purchase licenses directly from NetApp.

With the pay-as-you-go option, the Cloud Sync service charges hourly based on the number of sync relationships that you create. For pricing details, go to the Cloud Sync service page. To use this option, you must subscribe to the Cloud Sync service from the AWS Marketplace.

The other option is to pay for your sync relationships up front by purchasing licenses. Each license enables you to create up to 20 sync relationships. After you purchase a license, you must add it to Cloud Sync. For further details about licenses, see How Cloud Sync licenses work.

You can use both options together. For example, if you have 25 sync relationships, you can pay for the first 20 sync relationships using a license and then pay-as-you-go with the remaining 5 sync relationships.

What if I don’t immediately pay after my free trial ends?

You won’t be able to create any additional relationships. Existing relationships are not deleted, but you cannot make any changes to them until you subscribe or enter a license.

Subscribing to the Cloud Sync service from the AWS Marketplace

You can subscribe to the Cloud Sync service from the AWS Marketplace to pay for sync relationships at an hourly rate.

Steps
  1. Go to the License Settings page.

  2. Click Subscribe and then click Continue.

  3. Subscribe from the AWS Marketplace, and then log back in to the Cloud Sync service to complete the registration.

    The following video shows the process:

Purchasing licenses and adding them to the Cloud Sync service

To pay for your sync relationships up front, you must purchase one or more licenses and add them to the Cloud Sync service.

Steps
  1. Purchase a license by contacting NetApp.

  2. Go to the License Settings page and add the license.