Skip to main content

Fail over to an older snapshot using Trident Protect

Contributors netapp-shwetav
PDFs

Use Trident Protect to fail over a replicated application to an earlier snapshot, allowing recovery from a point in time before data was damaged or compromised. By default, a failover uses the most recently transferred snapshot.

Caution Failing over to an older snapshot causes ONTAP SnapMirror to delete the snapshots on the destination cluster that are newer than the selected snapshot, because they are no longer valid. Add the annotation in this procedure only if this data loss is acceptable.
Before you begin

  • An AppMirrorRelationship exists and is in the Established state. To create one, refer to Set up a replication relationship. To verify the state, run kubectl get amr -n <namespace> <relationship_name> -o=jsonpath='{.status}'.

  • The source and destination clusters use ONTAP storage backends with SnapMirror replication. Azure NetApp Files and other non-ONTAP backends are not supported. Refer to Requirements for SnapMirror replication.

  • A snapshot schedule (with snapshotRetention greater than 1) exists on the source application.

Identify the snapshot

Each snapshot has a completion time, so you can pick the point in time you want.

Steps

  1. On the source cluster, list the snapshots:

    kubectl get snapshots -n my-app-namespace

  2. Find the snapshot to fail over to. Use the completionTimestamp value to select the point in time.

  3. Get the archive path of that snapshot:

    kubectl get snapshots <snapshot_name> -n my-app-namespace -o jsonpath='{.status.appArchivePath}'

  4. Record the last part of the path, which is everything after the last slash. This part is the basename. For example:

    20231114173500_schedule-20231114173500_f8df8bd3-a727-4a14-b954-9666aa618b55
    Note Use the basename, not the name of the snapshot custom resource.
Result

The basename of the selected snapshot is recorded.

Fail over to the selected snapshot

Specify the snapshot to promote, and acknowledge the data loss by adding an annotation.

Steps

  1. On the destination cluster, add the annotation to the AppMirrorRelationship. Without the annotation, a failover can use only the most recently transferred snapshot.

    kubectl annotate appmirrorrelationship <relationship_name> -n my-app-namespace protect.trident.netapp.io/allow-promotion-with-older-snapshots="true"

  2. Edit the AppMirrorRelationship CR file (for example, trident-protect-relationship.yaml):

    • Set spec.desiredState to Promoted.

    • Set spec.promotedSnapshot to the basename you recorded in Identify the snapshot.

      For example:

      apiVersion: protect.trident.netapp.io/v1
kind: AppMirrorRelationship
metadata:
  name: <relationship_name>
  namespace: <namespace>
  annotations:
    protect.trident.netapp.io/allow-promotion-with-older-snapshots: "true"
spec:
  desiredState: Promoted
  promotedSnapshot: <basename>

  3. Apply the CR:

    kubectl apply -f trident-protect-relationship.yaml -n my-app-namespace

  4. Check the status of the relationship:

    kubectl get amr -n my-app-namespace <relationship_name> -o=jsonpath='{.status}' | jq
Result

The relationship moves to the Promoted state, and the application runs on the destination cluster with the data from the selected snapshot.

If the relationship stays in the Promoting state and status.error shows The snapshot provided is invalid or older than the most recently transferred snapshot, the annotation is missing or is not set to true. Add the annotation and try again.

What's next?

To return to the original cluster, perform a reverse resync, and then reverse the replication direction.

Caution To return to the original cluster, do not perform a normal resync. A normal resync discards the failed-over application and returns the relationship to the newest source snapshot.