Perform remote replication between clusters

Contributors netapp-dbagwell netapp-pcarriga amgrissino Download PDF of this page

For clusters running NetApp Element software, real-time replication enables the quick creation of remote copies of volume data. You can pair a storage cluster with up to four other storage clusters.

You can replicate volume data synchronously or asynchronously from either cluster in a cluster pair for failover and failback scenarios. You must first pair two NetApp Element clusters and then pair volumes on each cluster to take advantage of real-time remote replication.

What you’ll need
  • Ensure that you have added at least one cluster to the plug-in.

  • Ensure that all node IP addresses on both management and storage networks for paired clusters are routed to each other.

  • Ensure that the MTU of all paired nodes are the same and be supported end-to-end between clusters.

  • Ensure that the difference between NetApp Element software versions on the clusters is no greater than one major version. If the difference is greater, one of the clusters must be upgraded to perform data replication.

Note WAN Accelerator appliances have not been qualified by NetApp for use when replicating data. These appliances can interfere with compression and deduplication if deployed between two clusters that are replicating data. Be sure to fully qualify the effects of any WAN Accelerator appliance before you deploy it in a production environment.

Pair clusters

You must pair two clusters as a first step to using real-time replication functionality. After you pair and connect two clusters, you can configure active volumes on one cluster to be continuously replicated to a second cluster, providing continuous data protection (CDP).

You can pair a source and target cluster using the MVIP of the target cluster if there is Cluster Admin access to both clusters. If Cluster Admin access is only available on one cluster in a cluster pair, a pairing key can be used on the target cluster to complete the cluster pairing.

What you’ll need
  • You need Cluster Admin privileges to one or both clusters being paired.

  • Ensure there is less than 2000 ms of round-trip latency between clusters.

  • Ensure that the difference between NetApp Element software versions on the clusters is no greater than one major version.

  • Ensure that all node IPs on paired clusters are routed to each other.

Note Cluster pairing requires full connectivity between nodes on the management network. Replication requires connectivity between the individual nodes on the storage cluster network.

You can pair one NetApp Element cluster with up to four other clusters for replicating volumes. You can also pair clusters within the cluster group with each other.

Choose one of the following methods:

Pair clusters using known credentials

You can pair two clusters for real-time replication by using the MVIP of one cluster to establish a connection with the other cluster. Cluster Admin access on both clusters is required to use this method.

About this task

The Cluster Admin user name and password is used to authenticate cluster access before the clusters can be paired.

If the MVIP is not known, or access to the cluster is not available, you can pair the cluster by generating a pairing key and use the key to pair the two clusters. For instructions, see Pair clusters with a pairing key.

Steps
  1. From the vCenter that contains the local cluster, select NetApp Element Management > Protection.

    Note If two or more clusters are added, ensure that the cluster you intend to use for the task is selected in the navigation bar.
  2. Select the Cluster Pairs sub-tab.

  3. Select Create Cluster Pairing.

  4. Select one of the following:

    • Registered Cluster: If the remote cluster of the pairing is controlled by the same instance of the NetApp Element Plug-in for vCenter Server, select this.

    • Credentialed Cluster: If the remote cluster has known credentials that are outside of the NetApp Element Plug-in for vCenter Server configuration, select this.

  5. If you selected Registered Cluster, select a cluster from the list of available clusters and click Pair.

  6. If you selected Credentialed Cluster, do the following:

    1. Enter the remote cluster MVIP address.

    2. Enter a cluster administrator user name.

    3. Enter a cluster administrator password.

    4. Select Start Pairing.

  7. After the task completes and you see the Cluster Pairs page, verify that the cluster pair is connected.

  8. (Optional) On the remote cluster, select NetApp Element Management > Protection > Cluster Pairs or use the Element web UI to verify that the cluster pair is connected.

Pair clusters with a pairing key

If you have Cluster Admin access to a local cluster but not the remote cluster, you can pair the clusters using a pairing key. A pairing key is generated on a local cluster and then sent securely to a Cluster Admin at a remote site to establish a connection and complete the cluster pairing for real-time replication.

This procedure describes cluster pairing between two clusters using vCenter on the local and remote sites. For clusters not controlled by the vCenter Plug-in, you can alternatively start or complete cluster pairing using the Element web UI.

Steps
  1. From the vCenter that contains the local cluster, select NetApp Element Management > Protection.

    Note If two or more clusters are added, ensure that the cluster you intend to use for the task is selected in the navigation bar.
  2. Select the Cluster Pairs sub-tab.

  3. Select Create Cluster Pairing.

  4. Select Inaccessible Cluster.

  5. Select Generate Key.

    Note This action generates a text key for pairing and creates an unconfigured cluster pair on the local cluster. If you do not complete the procedure, you will need to manually delete the cluster pair.
  6. Copy the cluster pairing key to your clipboard.

  7. Select Close.

  8. Make the pairing key accessible to the Cluster Admin at the remote cluster site.

    Note The cluster pairing key contains a version of the MVIP, user name, password, and database information to permit volume connections for remote replication. This key should be treated in a secure manner and not stored in a way that would allow accidental or unsecured access to the user name or password.
    Important Do not modify any of the characters in the pairing key. The key becomes invalid if it is modified.
  9. From the vCenter that contains the remote cluster, select NetApp Element Management > Protection.

    Note If two or more clusters are added, ensure that the cluster you intend to use for the task is selected in the navigation bar.
    Note You can alternatively complete the pairing using the Element UI.
  10. Select the Cluster Pairs sub-tab.

  11. Select Complete Cluster Pairing.

    Note Wait for the loading spinner to disappear before proceeding to the next step. If an unexpected error occurs during the pairing process, check for and manually delete any unconfigured cluster pairs on the local and remote cluster and perform the pairing again.
  12. Paste the pairing key from the local cluster in the Cluster Pairing Key field.

  13. Select Pair Cluster.

  14. After the task completes and you see the Cluster Pair page, verify that the cluster pair is connected.

  15. To verify that the cluster pair is connected, on the remote cluster select NetApp Element Management > Protection or use the Element UI.

Validate the cluster pair connections

After the cluster pairing has completed, you might want to verify the cluster pair connection to ensure replication success.

Steps
  1. On the local cluster, select Data Protection > Cluster Pairs.

  2. Verify that the cluster pair is connected.

  3. Navigate back to the local cluster and the Cluster Pairs window and verify that the cluster pair is connected.

Pair volumes

After you have established a connection between clusters in a cluster pair, you can pair a volume on one cluster with a volume on the other cluster in the pair.

You can pair the volume using one of the following methods:

After a volume pairing relationship is established, you must identify which volume is the replication target:

What you’ll need
  • You should have established a connection between clusters in a cluster pair.

  • You need to have Cluster admin privileges to one or both clusters being paired.

Pair volumes using known credentials

You can pair a local volume with another volume on a remote cluster. Use this method if there is Cluster Admin access to both clusters on which volumes are to be paired. This method uses the volume ID of the volume on the remote cluster to initiate a connection.

Before you begin
  • You have Cluster Admin credentials for the remote cluster.

  • Ensure that the clusters containing the volumes are paired.

  • You know the remote Volume ID unless you intend to create a new volume during this process.

  • If you intend for the local volume to be the source, ensure that the access mode of the volume is set to Read/Write.

Steps
  1. From the vCenter that contains the local cluster, select NetApp Element Management > Management.

    Note If two or more clusters are added, ensure that the cluster you intend to use for the task is selected in the navigation bar.
  2. Select the Volumes sub-tab.

  3. From the Active view, select the check box for the volume that you want to pair.

  4. Select Actions.

  5. Select Volume Pairing.

  6. Select one of the following:

    • Volume Creation: To create a replication target volume on the remote cluster, select this. This method can be used only on remote clusters that are controlled by a NetApp Element Plug-in for vCenter Server.

    • Volume Selection: If the remote cluster for the target volume is controlled by a NetApp Element Plug-in for vCenter Server, select this.

    • Volume ID: If the remote cluster for the target volume has known credentials that are outside of the NetApp Element Plug-in for vCenter Server configuration, select this.

  7. Select a Replication Mode:

    • Real-time (Synchronous): Writes are acknowledged to the client after they are committed on both of the source and target clusters.

    • Real-time (Asynchronous): Writes are acknowledged to the client after they are committed on the source cluster.

    • Snapshots Only: Only snapshots created on the source cluster are replicated. Active writes from the source volume are not replicated.

  8. If you selected Volume Creation as the pairing mode option, do the following:

    1. Select a paired cluster from the drop-down list.

      Note This action populates the available accounts on the cluster to be selected in the next step.
    2. Select an account on the target cluster for the replication target volume.

    3. Enter a replication target volume name.

      Note Volume size cannot be adjusted during this process.
  9. If you selected Volume Selection as the pairing mode option, do the following:

    1. Select a paired cluster.

      Note This action populates the available volumes on the cluster to be selected in the next step.
    2. (Optional) Select the Set remote volume to Replication Target option if you want to set the remote volume as the target in the volume pairing. The local volume, if set to read/write, becomes the source in the pair.

      Important If you assign an existing volume as the replication target, the data on that volume will be overwritten. As a best practice, you should use a new volume as the replication target.
      Note You can also assign replication source and target later in the pairing process from Volumes > Actions > Edit. You must assign a source and target to complete the pairing.
    3. Select a volume from the list of available volumes.

  10. If you selected Volume ID as the pairing mode option, do the following:

    1. Select a paired cluster from the drop-down list.

    2. If the cluster is not registered with the plug-in, enter a cluster administrator user ID and a cluster administrator password.

    3. Enter a volume ID.

    4. Select the Set remote volume to Replication Target option if you want to set the remote volume as the target in the volume pairing. The local volume, if set to read/write, becomes the source in the pair.

      Important If you assign an existing volume as the replication target, the data on that volume will be overwritten. As a best practice, you should use a new volume as the replication target.
      Note You can also assign replication source and target later in the pairing process from Volumes > Actions > Edit. You must assign a source and target to complete the pairing.
  11. Select Pair.

    Note After you confirm the pairing, the two clusters begin the process of connecting the volumes. During the pairing process, you can see progress messages in the Volume Status column on the Volume Pairs page.
    Note If you have not yet assigned a volume to be the replication target, the pairing configuration is not complete. The volume pair displays PausedMisconfigured until the volume pair source and target are assigned. You must assign a source and target to complete the volume pairing.
  12. Select Protection > Volume Pairs on either cluster.

  13. Verify the status of the volume pairing.

Pair volumes using a pairing key

You can pair a local volume with another volume on a remote cluster using a pairing key. Use this method if there is Cluster Admin access to only the source cluster. This method generates a pairing key that can be used on the remote cluster to complete the volume pair.

Before you begin
  • Ensure that the clusters containing the volumes are paired.

  • Best Practices: Set the source volume to Read/Write and the target volume to Replication Target. The target volume should contain no data and have the exact characteristics of the source volume, such as size, 512e setting, and QoS configuration. If you assign an existing volume as the replication target, the data on that volume will be overwritten. The target volume may be greater or equal in size to the source volume, but it cannot be smaller.

About this task

This procedure describes volume pairing between two volumes using vCenter on the local and remote sites. For volumes not controlled by the vCenter Plug-in, you can alternately start or complete volume pairing using the Element web UI.

For instructions on starting or completing volume pairing from the Element web UI, see NetApp Element software documentation.

Note The volume pairing key contains an encrypted version of the volume information and may contain sensitive information. Share this key only in a secure manner.
Steps
  1. From the vCenter that contains the local cluster, select NetApp Element Management > Management.

    Note If two or more clusters are added, ensure that the cluster you intend to use for the task is selected in the navigation bar.
  2. Select the Volumes sub-tab.

  3. From the Active view, select the check box for the volume that you want to pair.

  4. Select Actions.

  5. Select Volume Pairing.

  6. Select Inaccessible Cluster.

  7. Select a Replication Mode:

    • Real-time (Synchronous): Writes are acknowledged to the client after they are committed on both of the source and target clusters.

    • Real-time (Asynchronous): Writes are acknowledged to the client after they are committed on the source cluster.

    • Snapshots Only: Only snapshots created on the source cluster are replicated. Active writes from the source volume are not replicated.

  8. Select Generate Key.

    Note This action generates a text key for pairing and creates an unconfigured volume pair on the local cluster. If you don’t do this, you will need to manually delete the volume pair.
  9. Copy the pairing key to your clipboard.

  10. Select Close.

  11. Make the pairing key accessible to the Cluster Admin at the remote cluster site.

    Note The volume pairing key should be treated in a secure manner and not stored in a way that would allow accidental or unsecured access.
    Important Do not modify any of the characters in the pairing key. The key becomes invalid if it is modified.
  12. From the vCenter that contains the remote cluster, select NetApp Element Management > Management.

    Note If two or more clusters are added, ensure that the cluster you intend to use for the task is selected in the navigation bar.
  13. Select the Volumes sub-tab.

  14. From the Active view, select the check box for the volume you want to pair.

  15. Select Actions.

  16. Select Volume Pairing.

  17. Select Complete Cluster Pairing.

  18. Paste the pairing key from the other cluster into the Pairing Key box.

  19. Select Complete Pairing.

    Note After you confirm the pairing, the two clusters begin the process of connecting the volumes. During the pairing process, you can see progress messages in the Volume Status column of the Volume Pairs page. If an unexpected error occurs during the pairing process, check for and manually delete any unconfigured cluster pairs on the local and remote cluster and perform the pairing again.
    Important If you have not yet assigned a volume to be the replication target, the pairing configuration is not complete. The volume pair displays "PausedMisconfigured" until the volume pair source and target are assigned. You must assign a source and target to complete the volume pairing.
  20. Select Protection > Volume Pairs on either cluster.

  21. Verify the status of the volume pairing.

    Note Volumes that are paired using a pairing key appear after the pairing process has been completed at the remote location.

Create target volumes and pair them with local volumes

You can pair two or more local volumes with associated target volumes on a remote cluster. This process creates a replication target volume on the remote cluster for each local source volume you select. Use this method if there is Cluster Admin access to both clusters on which volumes are to be paired and remote cluster is controlled by the plug-in.

This method uses the volume ID of each volume on the remote cluster to initiate one or more connections.

Before you begin
  • Ensure that you have Cluster Admin credentials for the remote cluster.

  • Ensure that the clusters containing the volumes are paired using the plug-in.

  • Ensure that the remote cluster is controlled by the plug-in.

  • Ensure that the access mode of each local volume is set to Read/Write.

Steps
  1. From the vCenter that contains the local cluster, select NetApp Element Management > Management.

    Note If two or more clusters are added, ensure that the cluster you intend to use for the task is selected in the navigation bar.
  2. Select the Volumes sub-tab.

  3. From the Active view, select two or more volumes that you want to pair.

  4. Select Actions.

  5. Select Volume Pairing.

  6. Select a Replication Mode:

    • Real-time (Synchronous): Writes are acknowledged to the client after they are committed on both of the source and target clusters.

    • Real-time (Asynchronous): Writes are acknowledged to the client after they are committed on the source cluster.

    • Snapshots Only: Only snapshots created on the source cluster are replicated. Active writes from the source volume are not replicated.

  7. Select a paired cluster from the drop-down list.

  8. Select an account on the target cluster for the replication target volume.

  9. (Optional) Type a prefix or suffix for the new volume names on the target cluster.

    Note A sample volume name with the modified name appears.
  10. Select Create Pairs.

    Note After you confirm the pairing, the two clusters begin the process of connecting the volumes. During the pairing process, you can see progress messages in the Volume Status column on the Volume Pairs page. After the process completes, new target volumes are created and connected on the remote cluster.
  11. Select Protection > Volume Pairs on either cluster.

  12. Verify the status of the volume pairing.

Assign a replication source and target to paired volumes

If you did not assign a volume to be the replication target during volume pairing, configuration is not complete. You can use this procedure to assign a source volume and its replication target volume. A replication source or target can be either volume in a volume pair.

You can also use this procedure to redirect data from a source volume to a remote target volume should the source volume become unavailable.

Before you begin

You have access to the clusters containing the source and target volumes.

About this task

This procedure describes assigning source and replication volumes between two clusters using vCenter on the local and remote sites. For volumes not controlled by the vCenter Plug-in, you can alternately assign a source or replication volume using the Element web UI.

A replication source volume has read/write account access. A replication target volume can only be accessed by the replication source as read/write.

Best Practices: The target volume should contain no data and have the exact characteristics of the source volume, such as size, 512e setting, and QoS configuration. The target volume may be greater or equal in size to the source volume, but it cannot be smaller.

Steps
  1. From the NetApp Element Management extension point in vCenter Server, select the cluster that contains the paired volume that you want to use as the replication source.

  2. Select NetApp Element Management > Management.

  3. Select the Volumes sub-tab.

  4. From the Active view, select the check box for the volume that you want to edit.

  5. Select Actions.

  6. Select Edit.

  7. From the Access drop-down list, select Read/Write.

    Important If you are reversing source and target assignment, this action will cause the volume pair to display PausedMisconfigured until a new replication target is assigned. Changing access pauses volume replication and causes the transmission of data to cease. Be sure that you have coordinated these changes at both sites.
  8. Select OK.

  9. From the NetApp Element Management extension point, select the cluster containing the paired volume that you want to use as the replication target.

  10. Select NetApp Element Management > Management.

  11. Select the Volumes sub-tab.

  12. From the Active view, select the check box for the volume you want to edit.

  13. Select Actions.

  14. Select Edit.

  15. In the Access drop-down list, select Replication Target.

    Important If you assign an existing volume as the replication target, the data on that volume will be overwritten. As a best practice, you should use a new volume as the replication target.
  16. Select OK.

Validate volume replication

After a volume is replicated, you should ensure that the source and target volumes are active. When in Active state, volumes are paired, data is being sent from the source to the target volume, and the data is in sync.

Steps
  1. From either cluster in the pairing, select NetApp Element Management > Protection.

    Note If two or more clusters are added, ensure that the cluster you intend to use for the task is selected in the navigation bar.
  2. Select the Volume Pairs sub-tab.

  3. Verify that the volume status is Active.

Delete a volume relationship after replication

After replication completes and you no longer need the volume pairing relationship, you can delete the volume relationship.

Manage volume relationships

You can manage volume relationships in many ways, such as pausing replication, reversing volume pairing, changing the mode of replication, deleting a volume pair, or deleting a cluster pair.

Pause replication

You can edit volume pair properties to manually pause replication.

Steps
  1. Select NetApp Element Management > Protection.

    Note If two or more clusters are added, ensure that the cluster you intend to use for the task is selected in the navigation bar.
  2. Select the Volume Pairs sub-tab.

  3. Select the check box for the volume pair you want to edit.

  4. Select Actions.

  5. Select Edit.

  6. Manually pause or start the replication process.

    Important Pausing or resuming volume replication manually will cause the transmission of data to cease or resume. Be sure that you have coordinated these changes at both sites.
  7. Select Save Changes.

Change the mode of replication

You can edit volume pair properties to make changes to the replication mode of the volume pair relationship.

Steps
  1. Select NetApp Element Management > Protection.

    Note If two or more clusters are added, ensure that the cluster you intend to use for the task is selected in the navigation bar.
  2. Select the Volume Pairs sub-tab.

  3. Select the check box for the volume pair you want to edit.

  4. Select Actions.

  5. Select Edit.

  6. Select a new replication mode:

    Important Changing the mode of replication causes the mode to change immediately. Be sure that you have coordinated these changes at both sites.
    • Real-time (Synchronous): Writes are acknowledged to the client after they are committed on both the source and target clusters.

    • Real-time (Asynchronous): Writes are acknowledged to the client after they are committed on the source cluster.

    • Snapshots Only: Only snapshots created on the source cluster are replicated. Active writes from the source volume are not replicated.

  7. Select Save Changes.

Delete a volume pair

You can delete a volume pair if you want to remove a pair association between two volumes.

About this task

This procedure describes deleting a volume pairing relationship between two volumes using vCenter on the local and remote sites.

For volumes not controlled by the vCenter Plug-in, you can alternatively delete a volume pair end using the Element web UI.

Steps
  1. Select NetApp Element Management > Protection.

    Note If two or more clusters are added, ensure that the cluster you intend to use for the task is selected in the navigation bar.
  2. Select the Volume Pairs sub-tab.

  3. Select one or more volume pairs you want to delete.

  4. Select Actions.

  5. Select Delete.

  6. Confirm the details of each volume pair.

    Note For clusters that are not managed by the plug-in, this action deletes only the volume pair end on the local cluster. You need to manually delete the volume pair end from the remote cluster to fully remove the pairing relationship.
  7. (Optional for clusters managed by plug-in) Select the check box for Change Replication Target Access to and select a new access mode for the replication target volume. This new access mode will be applied after the volume pairing relationship has been removed.

  8. Select Yes.

Delete a cluster pair

You can delete a cluster pairing relationship between two clusters using vCenter on the local and remote sites. To completely remove a cluster pairing relationship, you must remove cluster pair ends from both the local and remote clusters.

You can use the vCenter Plug-in to delete a cluster pair end.

For clusters not controlled by the vCenter Plug-in, you can alternatively delete a cluster pair end using the Element web UI.

Steps
  1. Select NetApp Element Management > Protection.

  2. Select the Cluster Pairs sub-tab.

  3. Select the check box for the cluster pair you want to delete.

  4. Select Actions.

  5. Select Delete.

  6. Confirm action.

    Note This action deletes only the cluster pair end on the local cluster. You need to manually delete the cluster pair end from the remote cluster to fully remove the pairing relationship.
  7. Repeat the steps from the remote cluster in the cluster pairing.

Volume pairing messages and warnings

You can view the information for volumes that have been paired or are in the process of being paired on the Volume Pairs page of the Protection tab from the NetApp Element Management configuration point.

The system displays pairing and progress messages in the Volume Status column.

Volume pairing messages

You can view messages during the initial pairing process on the Volume Pairs page of the Protection tab from the NetApp Element Management configuration point. These messages are displayed in the Volume Status column and can display on both source and target ends of the pairing.

  • PausedDisconnected: Source replication or sync RPCs timed out. Connection to the remote cluster has been lost. Check network connections to the cluster.

  • ResumingConnected*: The remote replication sync is now active. Beginning the sync process and waiting for data.

  • ResumingRRSync*: A single helix copy of the volume metadata is being made to the paired cluster.

  • ResumingLocalSync*: A double helix copy of the volume metadata is being made to the paired cluster.

  • ResumingDataTransfer*: Data transfer has been resumed.

  • Active: Volumes are paired and data is being sent from the source to the target volume and the data is in sync.

  • Idle: No replication activity is occurring.

*This process is driven by the target volume and might not display on the source volume.

Volume pairing warnings

You can view warning messages after you pair volumes on the Volume Pairs page of the Protection tab from the NetApp Element Management configuration point. These messages are displayed in the Volume Status column and can display on both source and target ends of the pairing.

These messages can display on both source and target ends of the pairing unless otherwise indicated.

  • PausedClusterFull: Because the target cluster is full, source replication and bulk data transfer cannot proceed. The message displays on the source end of the pair only.

  • PausedExceededMaxSnapshotCount: The target volume already has the maximum number of snapshots and cannot replicate additional snapshots.

  • PausedManual: Local volume has been manually paused. It must be unpaused before replication resumes.

  • PausedManualRemote: Remote volume is in manual paused mode. Manual intervention required to unpause the remote volume before replication resumes.

  • PausedMisconfigured: Waiting for an active source and target. Manual intervention required to resume replication.

  • PausedQoS: Target QoS could not sustain incoming IO. Replication auto-resumes. The message displays on the source end of the pair only.

  • PausedSlowLink: Slow link detected and stopped replication. Replication auto-resumes. The message displays on the source end of the pair only.

  • PausedVolumeSizeMismatch: Target volume is smaller than the source volume.

  • PausedXCopy: A SCSI XCOPY command is being issued to a source volume. The command must complete before replication can resume. The message displays on the source end of the pair only.

  • StoppedMisconfigured: A permanent configuration error has been detected. The remote volume has been purged or unpaired. No corrective action is possible; a new pairing must be established.

Find more information