Recover from automatic unplanned ONTAP cluster failover operations
Suggest changes
PDFs
An automatic unplanned failover (AUFO) operation occurs when the primary cluster is down or isolated. SnapMirror active sync detects such faults and initiates an automatic unplanned failover to the the secondary cluster. The secondary cluster is converted to the primary (the preferred cluster), and begins serving clients. Note that the failure detection and failover processing can only be performed with assistance from the ONTAP Mediator.
|
After the automatic unplanned failover, it is important to rescan the host paths so that there is no loss of I/O paths. |
Reestablish the protection relationship after an unplanned failover
Once the original primary cluster, for instance Cluster A, recovers, SnapMirror active sync automatically establishes protection in the reverse direction. If you want to restore the original state, making Cluster A the primary, preferred cluster, you can perform a manual failback by initiating a planned failover operation using System Manager or the ONTAP CLI.
|
Steps
From ONTAP 9.8 through ONTAP 9.14.1, SnapMirror active sync is referred to as SnapMirror Business Continuity (SM-BC).
|
-
Navigate to Protection > Relationships and wait for the relationship state to show "InSync".
-
To perform a failback and resume operations on the original source cluster, click
and select Failover.
You can monitor the status of the automatic unplanned failover using the snapmirror failover show command.
For example:
ClusterB::> snapmirror failover show -instance
Start Time: 9/23/2020 22:03:29
Source Path: vs1:/cg/scg3
Destination Path: vs3:/cg/dcg3
Failover Status: completed
Error Reason:
End Time: 9/23/2020 22:03:30
Primary Data Cluster: cluster-2
Last Progress Update: -
Failover Type: unplanned
Error Reason codes: -
You can monitor the status of the reverse protection relationship using the snapmirror show command on the original primary cluster.
For example:
ClusterA::> snapmirror show
Progress
Source Destination Mirror Relationship Total Last
Path Type Path State Status Progress Healthy Updated
----------- ---- ------------ ------- -------------- --------- ------- --------
vs3:/cg/dcg3 XDP vs1:/cg/scg3 Snapmirrored InSync - true -
You can perform a failback to the original primary cluster using the the snapmirror failover start command.
For example:
ClusterA::> snapmirror failover start -destination-path vs1:/cg/scg3
Refer to the EMS reference to learn about event messages and about corrective actions.
Resume protection in a fan-out configuration after failover
Beginning with ONTAP 9.15.1, SnapMirror active sync supports automatic reconfiguration in the fan-out leg after a failover event in a SAN relationship. In a NAS configuration, you must reconfigure the protection relationship manually after SnapMirror active sync failover. The async fan-out leg can be a consistency group relationship or an independent volume relationship. For more information, see fan-out configurations.
If you're using ONTAP 9.14.1 or earlier and you experience a failover on the secondary cluster in the SnapMirror active sync relationship, the SnapMirror asynchronous destination becomes unhealthy. You must manually restore protection by deleting and recreating the relationship with the SnapMirror asynchronous endpoint.
-
Verify the failover has completed successfully:
snapmirror failover show -
On the SnapMirror asynchronous endpoint, delete the fan-out endpoint:
snapmirror delete -destination-path destination_path -
On the third site, create a SnapMirror asynchronous relationships between the new SnapMirror active sync primary volume and the async fan-out destination volume:
snapmirror create -source-path source_path -destination-path destination_path -policy MirrorAllSnapshots -schedule schedule -
Resynchronize the relationship:
snapmirror resync -destination-path destination_path -
Verify the relationship status and heath:
snapmirror show