Uninstall Astra Data Store with an automated script

Contributors amgrissino netapp-dbagwell

To uninstall Astra Data Store and control plane, you will remove workloads, bindings, volumes, export policies, the Astra Data Store cluster, license, deployment environment, and the Astra Data Store namespace.

Uninstalling can be done using different methods:

Uninstall Astra Data Store with an automated script

This process uses an automated script to uninstall Astra Data Store.

What you’ll need
  • Root administrative permissions

About this task

The Astra Data Store uninstallation process guides you through the following high-level steps:

Remove existing workloads and bindings

Before uninstalling Astra Data Store, you must first remove the following

  • All application workloads that use Astra Data Store as the storage backend

  • Trident bindings that use Astra Data Store as a backend

This ensures that your Kubernetes environment is left in a clean state, which is important if you reinstall.

Uninstall Astra Data Store cluster

To uninstall Astra Data Store, you can use the uninstall.sh script in your Astra Data Store tar file that was downloaded from the NetApp Support Site.

  1. Locate the uninstall.sh in the manifests directory.

  2. Run the following sed command:

    sed -i -e 's~netappsdsoperator.yaml~astradsoperator.yaml~' uninstall.sh
  3. Run the following script indicating what you want to uninstall:

    ./uninstall.sh
    
    You must run this script with an argument specifying what should be uninstalled
    To uninstall the ADS cluster run ./uninstall.sh cluster
    To uninstall everything run ./uninstall all
  4. If you want to uninstall just the cluster, enter uninstall.sh <cluster>

    Otherwise, if you want to uninstall everything, enter uninstall.sh

    Note In most cases you will uninstall everything. You might want to uninstall just the cluster if you wanted to redeploy the cluster subsequently.
  5. At the prompt, confirm that you want to continue and enter erasedata

    Response:

    ./uninstall.sh all
    
    Enter 'erasedata' to confirm you want proceed with the uninstall: erasedata
    +----------------------------------------------------+
    | Wed Feb  2 10:14:01 EST 2022                       |
    | ADS cluster uninstall started                      |
    +----------------------------------------------------+
    Deleting astradsvolumes
    Deleted astradsvolumes
    Deleting astradsexportpolicies
    Deleted astradsexportpolicies
    Deleting astradsvolumesnapshots
    Deleted astradsvolumesnapshots
    Deleting astradsclusters
    Deleting astradsclusters
    Deleting astradslicenses
    Deleted astradslicenses
    
    +----------------------------------------------------+
    | Wed Feb  2 10:15:18 EST 2022                       |
    | ADS cluster uninstall done                         |
    +----------------------------------------------------+
    
    +----------------------------------------------------+
    | Wed Feb  2 10:15:18 EST 2022                       |
    | ADS system uninstall started                       |
    +----------------------------------------------------+
    Removing astradsversion
    astradsversion.astrads.netapp.io "astradsversion" deleted
    Removed astradsversion
    Removing daemonsets
    daemonset.apps "astrads-ds-nodeinfo-astradsversion" deleted
    Removed daemonsets
    Removing deployments
    deployment.apps "astrads-cluster-controller" deleted
    deployment.apps "astrads-license-controller" deleted
    deployment.apps "astrads-operator" deleted
    Removed deployments
    Removing all other AstraDS resources
    namespace "astrads-system" deleted
    customresourcedefinition.apiextensions.k8s.io "astradsautosupports.astrads.netapp.io" deleted
    customresourcedefinition.apiextensions.k8s.io "astradscloudsnapshots.astrads.netapp.io" deleted
    customresourcedefinition.apiextensions.k8s.io "astradsclusters.astrads.netapp.io" deleted
    customresourcedefinition.apiextensions.k8s.io "astradsexportpolicies.astrads.netapp.io" deleted
    customresourcedefinition.apiextensions.k8s.io "astradsfaileddrives.astrads.netapp.io" deleted
    customresourcedefinition.apiextensions.k8s.io "astradslicenses.astrads.netapp.io" deleted
    customresourcedefinition.apiextensions.k8s.io "astradsnfsoptions.astrads.netapp.io" deleted
    customresourcedefinition.apiextensions.k8s.io "astradsnodeinfoes.astrads.netapp.io" deleted
    customresourcedefinition.apiextensions.k8s.io "astradsnodemanagements.astrads.netapp.io" deleted
    customresourcedefinition.apiextensions.k8s.io "astradsqospolicies.astrads.netapp.io" deleted
    customresourcedefinition.apiextensions.k8s.io "astradsversions.astrads.netapp.io" deleted
    customresourcedefinition.apiextensions.k8s.io "astradsvolumefiles.astrads.netapp.io" deleted
    customresourcedefinition.apiextensions.k8s.io "astradsvolumes.astrads.netapp.io" deleted
    customresourcedefinition.apiextensions.k8s.io "astradsvolumesnapshots.astrads.netapp.io" deleted
    role.rbac.authorization.k8s.io "astrads-astrads-system-admin-role" deleted
    role.rbac.authorization.k8s.io "astrads-astrads-system-reader-role" deleted
    role.rbac.authorization.k8s.io "astrads-astrads-system-writer-role" deleted
    role.rbac.authorization.k8s.io "astrads-leader-election-role" deleted
    role.rbac.authorization.k8s.io "astrads-manager-role" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astrads-admin-clusterrole" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astrads-reader-clusterrole" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astrads-writer-clusterrole" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astradsautosupport-editor-role" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astradsautosupport-viewer-role" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astradscloudsnapshot-editor-role" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astradscloudsnapshot-viewer-role" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astradscluster-editor-role" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astradscluster-viewer-role" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astradsexportpolicy-editor-role" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astradsexportpolicy-viewer-role" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astradsfaileddrive-editor-role" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astradsfaileddrive-viewer-role" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astradslicense-editor-role" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astradslicense-viewer-role" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astradsnfsoption-editor-role" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astradsnfsoption-viewer-role" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astradsnodeinfo-editor-role" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astradsnodeinfo-viewer-role" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astradsnodemanagement-editor-role" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astradsnodemanagement-viewer-role" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astradsqospolicy-viewer-role" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astradsversion-editor-role" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astradsversion-viewer-role" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astradsvolume-editor-role" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astradsvolume-viewer-role" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astradsvolumefile-editor-role" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astradsvolumefile-viewer-role" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astradsvolumesnapshot-editor-role" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-astradsvolumesnapshot-viewer-role" deleted
    clusterrole.rbac.authorization.k8s.io "astrads-manager-role" deleted
    rolebinding.rbac.authorization.k8s.io "astrads-astrads-admin-rolebinding" deleted
    rolebinding.rbac.authorization.k8s.io "astrads-astrads-reader-rolebinding" deleted
    rolebinding.rbac.authorization.k8s.io "astrads-astrads-writer-rolebinding" deleted
    rolebinding.rbac.authorization.k8s.io "astrads-leader-election-rolebinding" deleted
    rolebinding.rbac.authorization.k8s.io "astrads-manager-rolebinding" deleted
    clusterrolebinding.rbac.authorization.k8s.io "astrads-astrads-admin-rolebinding" deleted
    clusterrolebinding.rbac.authorization.k8s.io "astrads-astrads-reader-rolebinding" deleted
    clusterrolebinding.rbac.authorization.k8s.io "astrads-astrads-writer-rolebinding" deleted
    clusterrolebinding.rbac.authorization.k8s.io "astrads-manager-rolebinding" deleted
    configmap "astrads-autosupport-cm" deleted
    configmap "astrads-firetap-cm" deleted
    configmap "astrads-kevents-asup" deleted
    configmap "astrads-metrics-cm" deleted
    secret "astrads-autosupport-certs" deleted
    +----------------------------------------------------+
    | Wed Feb  2 10:16:36 EST 2022                       |
    | ADS system uninstall done                          |
    +----------------------------------------------------+

Validate the removal of the astrads-system namespace

Ensure that the following command returns no result:

kubectl get ns | grep astrads-system

Ensure containers are not running on worker nodes

Validate that containers such as firetap or netwd are not running on the worker nodes.
Run the following on each node.

ssh <mynode1>
# runc list

Delete OpenShift Container Platform resources

If you installed Astra Data Store on Red Hat OpenShift Container Platform (OCP), you can uninstall OCP security context constraints (SCC) and rolebindings resources.

OpenShift uses security context constraints (SCC) that control the actions that a pod can perform.

After you complete the standard uninstall process, complete these steps.

  1. Remove SCC resources:

    oc delete -f ads_privileged_scc.yaml
  2. Remove rolebindings resources:

    oc delete -f oc_role_bindings.yaml
    Note Ignore "resources not found" errors in these steps. 

Uninstall Astra Data Store manually without a script

This process uninstalls Astra Data Store manually without a script.

To uninstall Astra Data Store manually without an automated script, you will remove workloads, bindings, volumes, export policies, clusters, license, deployment environment, and the Astra Data Store namespace.

What you’ll need
  • Root administrative permissions

About this task

The Astra Data Store uninstallation process guides you through the following high-level steps:

Remove existing workloads and bindings

Before uninstalling Astra Data Store, you must first remove the following

  • All application workloads that use Astra Data Store as the storage backend

  • Trident bindings that use Astra Data Store as a backend

This ensures that your Kubernetes environment is left in a clean state, which is important if you reinstall.

Uninstall the Astra Data Store cluster and control plane

Follow the steps below to uninstall Astra Data Store manually.

Delete the volumes and export policies

Before deleting the cluster, you should delete the Astra Data Store volumes and export policy.

Tip If you do not first delete volumes and export policies, the cluster deletion process pauses until the Astra Data Store volumes objects are deleted. It is more efficient to remove those items before starting to delete the cluster.
Steps
  1. Delete the volumes:

    ~% kubectl delete astradsvolumes --all -A
    ~% kubectl get astradsvolumes -A
  2. Delete the export policies:

    ~% kubectl delete astradsexportpolicies --all -A
    ~% kubectl get astradsexportpolicies -A

Delete the Astra Data Store cluster

Deleting the cluster deletes only the Astra Data Store cluster object custom resource (CR) along with cluster-scoped resources.

Note The operator, nodeinfo pods, and the cluster controller (which are Kubernetes-scoped resources) remain even after the cluster is deleted.

Deleting the cluster also uninstalls the underlying operating system from the nodes, which will stop the firetap and netwd services.

The uninstaller takes about a minute to finish. Then, the removal of the Astra Data Store cluster-scoped resources starts.

  1. Delete the cluster:

    ~% kubectl delete astradsclusters --all -A
    ~% kubectl get astradsclusters -A

Delete the license

  1. ssh to each worker node in the cluster and validate that firetap or netwd are not running in the worker nodes.

  2. Delete the Astra Data Store license:

    ~% kubectl delete astradslicenses --all -A
    ~% kubectl get astradslicenses -A

Delete the Astra Data Store installation

Delete the controllers, operators, namespace, and support pods in the cluster.

  1. Delete the Astra Data Store installation object:

    ~% kubectl delete astradsversion astradsversion -n astrads-system
    ~% kubectl get astradsversion -n astrads-system
  2. Delete the data store DaemonSets and all Astra Data Store controller resources:

    ~% kubectl delete ds --all -n astrads-system
    ~% kubectl get ds -n astrads-system
    
    ~% kubectl delete deployments --all -n astrads-system
    ~% kubectl get deployments -n astrads-system
  3. Delete remaining artifacts and the operator yaml file:

    ~% kubectl delete -f ./manifests/astradsoperator.yaml
    ~% kubectl get pods -n astrads-system

Validate the removal of the astrads-system namespace

Ensure that the following command returns no result:

~% kubectl get ns | grep astrads-system

Ensure containers are not running on worker nodes

Validate that containers such as firetap or netwd are not running on the worker nodes.
Run the following on each node.

ssh <mynode1>
# runc list

Delete OpenShift Container Platform resources

If you installed Astra Data Store on Red Hat OpenShift Container Platform (OCP), you can uninstall OCP security context constraints (SCC) and rolebindings resources.

OpenShift uses security context constraints (SCC) that control the actions that a pod can perform.

After you complete the standard uninstall process, complete these steps.

  1. Remove SCC resources:

    oc delete -f ads_privileged_scc.yaml
  2. Remove rolebindings resources:

    oc delete -f oc_role_bindings.yaml
    Note Ignore "resources not found errors" in these steps. 

Manual deletion sample

The following shows a sample of an execution manual uninstallation script.

$ kubectl delete astradsvolumes --all -A
No resources found
$ kubectl delete astradsexportpolicies --all -A
No resources found
$ kubectl delete astradsclusters --all -A
astradscluster.astrads.netapp.io "astrads-sti-c6220-09-10-11-12" deleted

$ kubectl delete astradslicenses --all -A
astradslicense.astrads.netapp.io "e900000005" deleted

$ kubectl delete astradsdeployment astradsdeployment -n astrads-system
astradsdeployment.astrads.netapp.io "astradsdeployment" deleted

$ kubectl delete ds --all -n astrads-system
daemonset.apps "astrads-ds-astrads-sti-c6220-09-10-11-12" deleted
daemonset.apps "astrads-ds-nodeinfo-astradsdeployment" deleted
daemonset.apps "astrads-ds-support" deleted

$ kubectl delete deployments --all -n astrads-system
deployment.apps "astrads-cluster-controller" deleted
deployment.apps "astrads-deployment-support" deleted
deployment.apps "astrads-license-controller" deleted
deployment.apps "astrads-operator" deleted

$ kubectl delete -f /.../firetap/sds/manifests/netappsdsoperator.yaml
namespace "astrads-system" deleted
customresourcedefinition.apiextensions.k8s.io "astradsautosupports.astrads.netapp.io" deleted
customresourcedefinition.apiextensions.k8s.io "astradscloudsnapshots.astrads.netapp.io" deleted
customresourcedefinition.apiextensions.k8s.io "astradsclusters.astrads.netapp.io" deleted
customresourcedefinition.apiextensions.k8s.io "astradsdeployments.astrads.netapp.io" deleted
customresourcedefinition.apiextensions.k8s.io "astradsexportpolicies.astrads.netapp.io" deleted
customresourcedefinition.apiextensions.k8s.io "astradsfaileddrives.astrads.netapp.io" deleted
customresourcedefinition.apiextensions.k8s.io "astradslicenses.astrads.netapp.io" deleted
customresourcedefinition.apiextensions.k8s.io "astradsnfsoptions.astrads.netapp.io" deleted
customresourcedefinition.apiextensions.k8s.io "astradsnodeinfoes.astrads.netapp.io" deleted
customresourcedefinition.apiextensions.k8s.io "astradsqospolicies.astrads.netapp.io" deleted
customresourcedefinition.apiextensions.k8s.io "astradsvolumefiles.astrads.netapp.io" deleted
customresourcedefinition.apiextensions.k8s.io "astradsvolumes.astrads.netapp.io" deleted
customresourcedefinition.apiextensions.k8s.io "astradsvolumesnapshots.astrads.netapp.io" deleted
role.rbac.authorization.k8s.io "astrads-leader-election-role" deleted
clusterrole.rbac.authorization.k8s.io "astrads-astradscloudsnapshot-editor-role" deleted
clusterrole.rbac.authorization.k8s.io "astrads-astradscloudsnapshot-viewer-role" deleted
clusterrole.rbac.authorization.k8s.io "astrads-astradscluster-editor-role" deleted
clusterrole.rbac.authorization.k8s.io "astrads-astradscluster-viewer-role" deleted
clusterrole.rbac.authorization.k8s.io "astrads-astradslicense-editor-role" deleted
clusterrole.rbac.authorization.k8s.io "astrads-astradslicense-viewer-role" deleted
clusterrole.rbac.authorization.k8s.io "astrads-astradsvolume-editor-role" deleted
clusterrole.rbac.authorization.k8s.io "astrads-astradsvolume-viewer-role" deleted
clusterrole.rbac.authorization.k8s.io "astrads-autosupport-editor-role" deleted
clusterrole.rbac.authorization.k8s.io "astrads-autosupport-viewer-role" deleted
clusterrole.rbac.authorization.k8s.io "astrads-manager-role" deleted
clusterrole.rbac.authorization.k8s.io "astrads-metrics-reader" deleted
clusterrole.rbac.authorization.k8s.io "astrads-netappexportpolicy-editor-role" deleted
clusterrole.rbac.authorization.k8s.io "astrads-netappexportpolicy-viewer-role" deleted
clusterrole.rbac.authorization.k8s.io "astrads-netappsdsdeployment-editor-role" deleted
clusterrole.rbac.authorization.k8s.io "astrads-netappsdsdeployment-viewer-role" deleted
clusterrole.rbac.authorization.k8s.io "astrads-netappsdsnfsoption-editor-role" deleted
clusterrole.rbac.authorization.k8s.io "astrads-netappsdsnfsoption-viewer-role" deleted
clusterrole.rbac.authorization.k8s.io "astrads-netappsdsnodeinfo-editor-role" deleted
clusterrole.rbac.authorization.k8s.io "astrads-netappsdsnodeinfo-viewer-role" deleted
clusterrole.rbac.authorization.k8s.io "astrads-proxy-role" deleted
rolebinding.rbac.authorization.k8s.io "astrads-leader-election-rolebinding" deleted
clusterrolebinding.rbac.authorization.k8s.io "astrads-manager-rolebinding" deleted
clusterrolebinding.rbac.authorization.k8s.io "astrads-proxy-rolebinding" deleted
configmap "astrads-autosupport-cm" deleted
configmap "astrads-firetap-cm" deleted
configmap "astrads-fluent-bit-cm" deleted
configmap "astrads-kevents-asup" deleted
configmap "astrads-metrics-cm" deleted
service "astrads-operator-metrics-service" deleted
 Error from server (NotFound): error when deleting "/.../export/firetap/sds/manifests/netappsdsoperator.yaml": deployments.apps "astrads-operator" not found

$ kubectl get ns | grep astrads-system

[root@sti-rx2540-535c ~]# runc list
ID      PID     STATUS    BUNDLE       CREATED    OWNER

Troubleshoot the Astra Data Store uninstall process

If you need to troubleshoot the uninstall process, review the following suggestions.

Pods in terminating state

The Astra Data Store uninstall process can occasionally cause pods to remain in a terminating state in Kubernetes.

If this issue occurs, run the following command to force delete all pods in the astrads-system namespace:

kubectl delete pods --all  -n astrads-system   --force --grace-period 0

Quality of Service policy points to old cluster

If you delete the Astra Data Store Cluster only and redeploy it, you might not be able to create a persistent volume claim (PVC) or volume because the Quality of Service (QoS) policy points to the old cluster and cannot be found.

  1. To avoid this, after deleting the Astra Data Store cluster, manually delete the QoS policy:

    kubectl delete AstraDSQosPolicy --all -A
  2. Delete the entire Astra Data Store deployment (not just the cluster):

    uninstall.sh all

Key provider CRs not removed after deleting or uninstalling Astra Data Store

If an external key provider has been configured for an Astra Data Store cluster that is being deleted or uninstalled, you might need to manually clean up any key provider CR that was not removed.

Details

Use the following workaround instructions:

Steps
  1. Confirm that the key provider CRs were not removed:

    kubectl get astradskeyprovider --selector astrads.netapp.io/cluster=astrads-cluster-example -n astrads-system

    Response:

    NAME                   AGE
    externalkeyprovider1   94s
  2. Remove the key provider CRs:

    1. Remove the finalizer:

      kubectl edit astradskeyprovider -n astrads-system
    2. Remove the finalizer line highlighted below:

      kubectl edit astradskeyprovider externalkeyprovider1 -n astrads-system
      apiVersion: astrads.netapp.io/v1beta1
      kind: AstraDSKeyProvider
      metadata:
        creationTimestamp: "2022-05-24T16:38:27Z"
        finalizers:
        - astrads.netapp.io/astradskeyprovider-finalizer
        generation: 1
        labels:
          astrads.netapp.io/cluster: astrads-cluster-example
          astrads.netapp.io/rsid: "1"
        name: externalkeyprovider1
        namespace: astrads-system
        resourceVersion: "1134699"
        uid: a11111b2-31c0-4575-b7f3-97f9abla1bla
      spec:
        cluster: astrads-cluster-example
        kmipServer:
          hostnames:
          - 10.xxx.xxx.xxx
          port: 5696
          secretRef: externalkeyprovider1
      status:
        keyProviderUUID: a1b2cd34-4fc6-5bae-9184-2288c673181d
        kmipServerStatus:
          capabilities: '{ KMIP_library_version()=17367809, KMIP_library_version_str()="KMIP
            1.9.3a  8-Apr-2019", KMIP_library_version_tag()="KMIP part of KMIP 1.9.3a  8-Apr-2019",
            KMIP_library_is_eval()=false, KMIP_library_fips_capable()=true(FIPS140), KMIP_SSL_provider_build_version()=268444095,
            KMIP_SSL_provider_version()=268444095, KMIP_SSL_provider_version_str()="OpenSSL
            1.0.2zb-fips  23 Sep 2021" }'
          keyServerUUID: 8422bdd0-74ad-579d-81bd-6d544ac4224a
    3. After the finalizer has been removed, delete the key provider CR:

      kubectl delete astradskeyprovider <key-provider-cr-name> -n astrads-system

Unable to uninstall Astra Data Store from Astra Control Center web UI

The Astra Data Store uninstallation process can occasionally fail if you initiated it from the Astra Control Center web UI.

If this issue occurs, take the following steps.

Steps
  1. Log in to the NetApp Support Site and download the Astra Data Store bundle (Astra_Data_Store_2022.05.01.tar) to a machine that can access the Kubernetes cluster where Astra Data Store resides.

  2. Log in to the machine to which you downloaded the Astra Data Store bundle.

  3. Extract the contents of the bundle:

    tar -xvf <path to tar file>/Astra_Data_Store_2022.05.01.tar
  4. Change to the manifests directory, where the uninstall script is stored:

    cd astrads/manifests/
  5. Manually remove Astra Data Store:

    ./uninstall all