Upgrade Astra Control Center

04/25/2022 Contributors

To upgrade Astra Control Center, download the installation bundle from the NetApp Support Site and complete these instructions to upgrade the Astra Control Center components in your environment. You can use this procedure to upgrade Astra Control Center in internet-connected or air-gapped environments.

What you'll need

Before you begin upgrade, ensure your environment still meets the minimum requirements for Astra Control Center deployment.
Ensure all cluster operators are in a healthy state and available.

OpenShift example:
```
oc get clusteroperators
```
Ensure all API services are in a healthy state and available.

OpenShift example:
```
oc get apiservices
```
Log out of your Astra Control Center.

About this task

The Astra Control Center upgrade process guides you through the following high-level steps:

Download the Astra Control Center bundle
Unpack the bundle and change directory
Add the images to your local registry
Install the updated Astra Control Center operator
Upgrade Astra Control Center
Upgrade third-party services
Verify system status

Do not execute the following command during the entirety of the upgrade process to avoid deleting all Astra Control Center pods: kubectl delete -f astra_control_center_operator_deploy.yaml

Perform upgrades in a maintenance window when schedules, backups, and snapshots are not running.

Podman commands can be used in place of Docker commands if you are using Red Hat’s Podman instead of Docker Engine.

Download the Astra Control Center bundle

Download the Astra Control Center upgrade bundle (astra-control-center-[version].tar.gz) from the NetApp Support Site.

(Optional) Use the following command to verify the signature of the bundle:

openssl dgst -sha256 -verify astra-control-center[version].pub -signature <astra-control-center[version].sig astra-control-center[version].tar.gz

Unpack the bundle and change directory

Extract the images:

tar -vxzf astra-control-center-[version].tar.gz

Change to the Astra directory.
```
cd astra-control-center-[version]
```

Add the images to your local registry

Add the files in the Astra Control Center image directory to your local registry.

See a sample script for the automatic loading of images below.

Log in to your Docker registry:
```
docker login [your_registry_path]
```
Load the images into Docker.
Tag the images.

Push the images to your local registry.

export REGISTRY=[your_registry_path]
for astraImageFile in $(ls images/*.tar)
  # Load to local cache. And store the name of the loaded image trimming the 'Loaded images: '
  do astraImage=$(docker load --input ${astraImageFile} | sed 's/Loaded image: //')
  astraImage=$(echo ${astraImage} | sed 's!localhost/!!')
  # Tag with local image repo.
  docker tag ${astraImage} ${REGISTRY}/${astraImage}
  # Push to the local repo.
  docker push ${REGISTRY}/${astraImage}
done

Install the updated Astra Control Center operator

Edit the Astra Control Center operator deployment yaml (astra_control_center_operator_deploy.yaml) to refer to your local registry and secret.

vim astra_control_center_operator_deploy.yaml

If you use a registry that requires authentication, replace the default line of imagePullSecrets: [] with the following:
```
imagePullSecrets:
- name: <name_of_secret_with_creds_to_local_registry>
```
Change [your_registry_path] for the kube-rbac-proxy image to the registry path where you pushed the images in a previous step.
Change [your_registry_path] for the acc-operator-controller-manager image to the registry path where you pushed the images in a previous step.

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    control-plane: controller-manager
  name: acc-operator-controller-manager
  namespace: netapp-acc-operator
spec:
  replicas: 1
  selector:
    matchLabels:
      control-plane: controller-manager
  template:
    metadata:
      labels:
        control-plane: controller-manager
    spec:
      containers:
      - args:
        - --secure-listen-address=0.0.0.0:8443
        - --upstream=http://127.0.0.1:8080/
        - --logtostderr=true
        - --v=10
        image: [your_registry_path]/kube-rbac-proxy:v4.8.0
        name: kube-rbac-proxy
        ports:
        - containerPort: 8443
          name: https
      - args:
        - --health-probe-bind-address=:8081
        - --metrics-bind-address=127.0.0.1:8080
        - --leader-elect
        command:
        - /manager
        env:
        - name: ACCOP_LOG_LEVEL
          value: "2"
        image: [your_registry_path]/acc-operator:[version x.y.z]
        imagePullPolicy: IfNotPresent
      imagePullSecrets: []

Install the updated Astra Control Center operator:

kubectl apply -f astra_control_center_operator_deploy.yaml

Sample response:

namespace/netapp-acc-operator unchanged
customresourcedefinition.apiextensions.k8s.io/astracontrolcenters.astra.netapp.io configured
role.rbac.authorization.k8s.io/acc-operator-leader-election-role unchanged
clusterrole.rbac.authorization.k8s.io/acc-operator-manager-role configured
clusterrole.rbac.authorization.k8s.io/acc-operator-metrics-reader unchanged
clusterrole.rbac.authorization.k8s.io/acc-operator-proxy-role unchanged
rolebinding.rbac.authorization.k8s.io/acc-operator-leader-election-rolebinding unchanged
clusterrolebinding.rbac.authorization.k8s.io/acc-operator-manager-rolebinding configured
clusterrolebinding.rbac.authorization.k8s.io/acc-operator-proxy-rolebinding unchanged
configmap/acc-operator-manager-config unchanged
service/acc-operator-controller-manager-metrics-service unchanged
deployment.apps/acc-operator-controller-manager configured

Upgrade Astra Control Center

Edit the Astra Control Center custom resource (CR) and change the Astra version (astraVersion inside of Spec) number to the latest:
```
kubectl edit acc -n [netapp-acc or custom namespace]
```
Changing the Astra version is the only requirement for an Astra Control Center upgrade. Your registry path must match the registry path where you pushed the images in a previous step.

Verify that the pods terminate and become available again:

watch kubectl get pods -n [netapp-acc or custom namespace]

Verify that all system components upgraded successfully.

kubectl get pods -n [netapp-acc or custom namespace]

Each pod should have a status of Running and Age that is recent. It may take several minutes before the system pods are deployed.

Sample response:

NAME                                         READY   STATUS    RESTARTS   AGE
acc-helm-repo-5f75c5f564-bzqmt             1/1     Running   0          11m
activity-6b8f7cccb9-mlrn4                  1/1     Running   0          9m2s
api-token-authentication-6hznt             1/1     Running   0          8m50s
api-token-authentication-qpfgb             1/1     Running   0          8m50s
api-token-authentication-sqnb7             1/1     Running   0          8m50s
asup-5578bbdd57-dxkbp                      1/1     Running   0          9m3s
authentication-56bff4f95d-mspmq            1/1     Running   0          7m31s
bucketservice-6f7968b95d-9rrrl             1/1     Running   0          8m36s
cert-manager-5f6cf4bc4b-82khn              1/1     Running   0          6m19s
cert-manager-cainjector-76cf976458-sdrbc   1/1     Running   0          6m19s
cert-manager-webhook-5b7896bfd8-2n45j      1/1     Running   0          6m19s
cloud-extension-749d9f684c-8bdhq           1/1     Running   0          9m6s
cloud-insights-service-7d58687d9-h5tzw     1/1     Running   2          8m56s
composite-compute-968c79cb5-nv7l4          1/1     Running   0          9m11s
composite-volume-7687569985-jg9gg          1/1     Running   0          8m33s
credentials-5c9b75f4d6-nx9cz               1/1     Running   0          8m42s
entitlement-6c96fd8b78-zt7f8               1/1     Running   0          8m28s
features-5f7bfc9f68-gsjnl                  1/1     Running   0          8m57s
fluent-bit-ds-h88p7                        1/1     Running   0          7m22s
fluent-bit-ds-krhnj                        1/1     Running   0          7m23s
fluent-bit-ds-l5bjj                        1/1     Running   0          7m22s
fluent-bit-ds-lrclb                        1/1     Running   0          7m23s
fluent-bit-ds-s5t4n                        1/1     Running   0          7m23s
fluent-bit-ds-zpr6v                        1/1     Running   0          7m22s
graphql-server-5f5976f4bd-vbb4z            1/1     Running   0          7m13s
identity-56f78b8f9f-8h9p9                  1/1     Running   0          8m29s
influxdb2-0                                1/1     Running   0          11m
krakend-6f8d995b4d-5khkl                   1/1     Running   0          7m7s
license-5b5db87c97-jmxzc                   1/1     Running   0          9m
login-ui-57b57c74b8-6xtv7                  1/1     Running   0          7m10s
loki-0                                     1/1     Running   0          11m
monitoring-operator-9dbc9c76d-8znck        2/2     Running   0          7m33s
nats-0                                     1/1     Running   0          11m
nats-1                                     1/1     Running   0          10m
nats-2                                     1/1     Running   0          10m
nautilus-6b9d88bc86-h8kfb                  1/1     Running   0          8m6s
nautilus-6b9d88bc86-vn68r                  1/1     Running   0          8m35s
openapi-b87d77dd8-5dz9h                    1/1     Running   0          9m7s
polaris-consul-consul-5ljfb                1/1     Running   0          11m
polaris-consul-consul-s5d5z                1/1     Running   0          11m
polaris-consul-consul-server-0             1/1     Running   0          11m
polaris-consul-consul-server-1             1/1     Running   0          11m
polaris-consul-consul-server-2             1/1     Running   0          11m
polaris-consul-consul-twmpq                1/1     Running   0          11m
polaris-mongodb-0                          2/2     Running   0          11m
polaris-mongodb-1                          2/2     Running   0          10m
polaris-mongodb-2                          2/2     Running   0          10m
polaris-ui-84dc87847f-zrg8w                1/1     Running   0          7m12s
polaris-vault-0                            1/1     Running   0          11m
polaris-vault-1                            1/1     Running   0          11m
polaris-vault-2                            1/1     Running   0          11m
public-metrics-657698b66f-67pgt            1/1     Running   0          8m47s
storage-backend-metrics-6848b9fd87-w7x8r   1/1     Running   0          8m39s
storage-provider-5ff5868cd5-r9hj7          1/1     Running   0          8m45s
telegraf-ds-dw4hg                          1/1     Running   0          7m23s
telegraf-ds-k92gn                          1/1     Running   0          7m23s
telegraf-ds-mmxjl                          1/1     Running   0          7m23s
telegraf-ds-nhs8s                          1/1     Running   0          7m23s
telegraf-ds-rj7lw                          1/1     Running   0          7m23s
telegraf-ds-tqrkb                          1/1     Running   0          7m23s
telegraf-rs-9mwgj                          1/1     Running   0          7m23s
telemetry-service-56c49d689b-ffrzx         1/1     Running   0          8m42s
tenancy-767c77fb9d-g9ctv                   1/1     Running   0          8m52s
traefik-5857d87f85-7pmx8                   1/1     Running   0          6m49s
traefik-5857d87f85-cpxgv                   1/1     Running   0          5m34s
traefik-5857d87f85-lvmlb                   1/1     Running   0          4m33s
traefik-5857d87f85-t2xlk                   1/1     Running   0          4m33s
traefik-5857d87f85-v9wpf                   1/1     Running   0          7m3s
trident-svc-595f84dd78-zb8l6               1/1     Running   0          8m54s
vault-controller-86c94fbf4f-krttq          1/1     Running   0          9m24s

Verify that the Astra status conditions indicate that the upgrade is complete and ready:

kubectl get -o yaml -n [netapp-acc or custom namespace] astracontrolcenters.astra.netapp.io astra

Response:

conditions:
  - lastTransitionTime: "2021-10-25T18:49:26Z"
    message: Astra is deployed
    reason: Complete
    status: "True"
    type: Ready
  - lastTransitionTime: "2021-10-25T18:49:26Z"
    message: Upgrading succeeded.
    reason: Complete
    status: "False"
    type: Upgrading

Upgrade third-party services

The third-party services Traefik and Cert-manager are not upgraded during earlier upgrade steps. You can optionally upgrade them using the procedure described here or retain existing service versions if your system requires it. The following is the recommended Traefik and Certs-manager upgrade sequence:

Set up acc-helm-repo to upgrade Traefik and Cert-manager
Update Traefik service using acc-helm-repo
Update the Cert-manager service

Set up acc-helm-repo to upgrade Traefik and Cert-manager

Find the enterprise-helm-repo that is loaded to your local Docker cache:

docker images enterprise-helm-repo

Response:

REPOSITORY             TAG         IMAGE ID       CREATED        SIZE
enterprise-helm-repo   21.10.218   7a182d6b30f3   20 hours ago   464MB

Start a container using the tag from the previous step:

docker run -dp 8082:8080 enterprise-helm-repo:21.10.218

Response:

940436e67fa86d2c4559ac4987b96bb35588313c2c9ddc9cec195651963f08d8

Add the Helm repo to your local host repositories:

helm repo add acc-helm-repo http://localhost:8082/

Response:

"acc-helm-repo" has been added to your repositories

Save the following Python script as a file, for example, set_previous_values.py:

This Python script creates two files that are used in later upgrade steps to retain helm values.

#!/usr/bin/env python3
import json
import os

NAMESPACE = "netapp-acc"

os.system(f"helm get values traefik -n {NAMESPACE} -o json > traefik_values.json")
os.system(f"helm get values cert-manager -n {NAMESPACE} -o json > cert_manager_values.json")

# reformat traefik values
f = open("traefik_values.json", "r")
traefik_values = {'traefik': json.load(f)}
f.close()

with open('traefik_values.json', 'w') as output_file:
    json.dump(traefik_values, output_file)

# reformat cert-manager values
f = open("cert_manager_values.json", "r")
cm_values = {'cert-manager': json.load(f)}
f.close()

cm_values['global'] = cm_values['cert-manager']['global']
del cm_values['cert-manager']['global']

with open('cert_manager_values.json', 'w') as output_file:
    json.dump(cm_values, output_file)

print('Done')

Run the script:
```
python3.7 ./set_previous_values.py
```

Update Traefik service using acc-helm-repo

You must already have set up acc-helm-repo before completing the following procedure.

Download the Traefik bundle using a secure, file-transfer tool, such as GNU wget:
```
wget http://localhost:8082/traefik-0.2.0.tgz
```
Extract the images:
```
tar -vxzf traefik-0.2.0.tgz
```

Apply the Traefik CRDs:

kubectl apply -f ./traefik/charts/traefik/crds/

Find the Helm chart version to use with your upgraded Traefik:

helm search repo acc-helm-repo/traefik

Response:

NAME                                    CHART VERSION   APP VERSION DESCRIPTION
acc-helm-repo/traefik                 0.2.0           2.5.3       Helm chart for Traefik Ingress controller
acc-helm-repo/traefik-ingressroutes   0.2.0           2.5.3       A Helm chart for Kubernetes

Validate the traefik_values.json file for upgrade:
1. Open the traefik_values.json file.
2. Check if there is a value for the imagePullSecret field. If it is empty, remove the following text from the file:
  "imagePullSecrets": [{"name": ""}],
3. Ensure that the traefik image is directed to the correct location and has the correct name:
  image: [your_registry_path]/traefik

Upgrade your Traefik configuration:

helm upgrade --version 0.2.0 --namespace netapp-acc -f traefik_values.json traefik acc-helm-repo/traefik

Response:

Release "traefik" has been upgraded. Happy Helming!
NAME: traefik
LAST DEPLOYED: Mon Oct 25 22:53:19 2021
NAMESPACE: netapp-acc
STATUS: deployed
REVISION: 2
TEST SUITE: None

Update the Cert-manager service

You must already have completed the Traefik update and added acc-helm-repo in Helm before completing the following procedure.

Find the helm chart version to use with your upgraded Cert-manager:

helm search repo acc-helm-repo/cert-manager

Response:

NAME CHART VERSION APP VERSION DESCRIPTION
acc-helm-repo/cert-manager 0.3.0 v1.5.4 A Helm chart for cert-manager
acc-helm-repo/cert-manager-certificates 0.1.0 1.16.0 A Helm chart for Kubernetes

Validate the cert_manager_values.json file for upgrade:
1. Open the cert_manager_values.json file.
2. Check if there is a value for the imagePullSecret field. If it is empty, remove the following text from the file:
  "imagePullSecrets": [{"name": ""}],
3. Ensure that the three cert-manager images are directed to the correct location and have the correct names.

Upgrade your Cert-manager configuration:

helm upgrade --version 0.3.0 --namespace netapp-acc -f cert_manager_values.json cert-manager acc-helm-repo/cert-manager

Response:

Release "cert-manager" has been upgraded. Happy Helming!
NAME: cert-manager
LAST DEPLOYED: Tue Nov 23 11:20:05 2021
NAMESPACE: netapp-acc
STATUS: deployed
REVISION: 2
TEST SUITE: None

Verify system status

Log in to Astra Control Center.
Verify that all your managed clusters and apps are still present and protected.

Upgrade Astra Control Center

Creating your file...

Download the Astra Control Center bundle

Unpack the bundle and change directory

Add the images to your local registry

Install the updated Astra Control Center operator

Upgrade Astra Control Center

Upgrade third-party services

Set up acc-helm-repo to upgrade Traefik and Cert-manager

Update Traefik service using acc-helm-repo

Update the Cert-manager service

Verify system status