Creating Private Image Registries

10/07/2024 Contributors

For most deployments of Red Hat OpenShift, using a public registry like Quay.io or DockerHub meets most customer's needs. However there are times when a customer may want to host their own private or customized images.

This procedure documents creating a private image registry which is backed by a persistent volume provided by Trident and NetApp ONTAP.

Astra Control Center requires a registry to host the images the Astra containers require. The following section describes the steps to setup a private registry on Red Hat OpenShift cluster and pushing the images required to support the installation of Astra Control Center.

Creating A private image registry

Remove the default annotation from the current default storage class and annotate the Trident-backed storage class as default for the OpenShift cluster.

[netapp-user@rhel7 ~]$ oc patch storageclass thin -p '{"metadata": {"annotations": {"storageclass.kubernetes.io/is-default-class": "false"}}}'
storageclass.storage.k8s.io/thin patched

[netapp-user@rhel7 ~]$ oc patch storageclass ocp-trident -p '{"metadata": {"annotations": {"storageclass.kubernetes.io/is-default-class": "true"}}}'
storageclass.storage.k8s.io/ocp-trident patched

Edit the imageregistry operator by entering the following storage parameters in the spec section.

[netapp-user@rhel7 ~]$ oc edit configs.imageregistry.operator.openshift.io

storage:
  pvc:
    claim:

Enter the following parameters in the spec section for creating a OpenShift route with a custom hostname. Save and exit.

routes:
  - hostname: astra-registry.apps.ocp-vmw.cie.netapp.com
    name: netapp-astra-route

The above route config is used when you want a custom hostname for your route. If you want OpenShift to create a route with a default hostname, you can add the following parameters to the spec section: defaultRoute: true.

Custom TLS certificates

When you are using a custom hostname for the route, by default, it uses the default TLS configuration of the OpenShift Ingress operator. However, you can add a custom TLS configuration to the route. To do so, complete the following steps.

Create a secret with the route’s TLS certificates and key.

[netapp-user@rhel7 ~]$ oc create secret tls astra-route-tls -n openshift-image-registry –cert/home/admin/netapp-astra/tls.crt --key=/home/admin/netapp-astra/tls.key

Edit the imageregistry operator and add the following parameters to the spec section.

[netapp-user@rhel7 ~]$ oc edit configs.imageregistry.operator.openshift.io

routes:
  - hostname: astra-registry.apps.ocp-vmw.cie.netapp.com
    name: netapp-astra-route
    secretName: astra-route-tls

Edit the imageregistry operator again and change the management state of the operator to the Managed state. Save and exit.
```
oc edit configs.imageregistry/cluster

managementState: Managed
```

If all the prerequisites are satisfied, PVCs, pods, and services are created for the private image registry. In a few minutes, the registry should be up.

[netapp-user@rhel7 ~]$oc get all -n openshift-image-registry

NAME                                                   READY   STATUS      RESTARTS   AGE
pod/cluster-image-registry-operator-74f6d954b6-rb7zr   1/1     Running     3          90d
pod/image-pruner-1627257600-f5cpj                      0/1     Completed   0          2d9h
pod/image-pruner-1627344000-swqx9                      0/1     Completed   0          33h
pod/image-pruner-1627430400-rv5nt                      0/1     Completed   0          9h
pod/image-registry-6758b547f-6pnj8                     1/1     Running     0          76m
pod/node-ca-bwb5r                                      1/1     Running     0          90d
pod/node-ca-f8w54                                      1/1     Running     0          90d
pod/node-ca-gjx7h                                      1/1     Running     0          90d
pod/node-ca-lcx4k                                      1/1     Running     0          33d
pod/node-ca-v7zmx                                      1/1     Running     0          7d21h
pod/node-ca-xpppp                                      1/1     Running     0          89d

NAME                              TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)     AGE
service/image-registry            ClusterIP   172.30.196.167   <none>        5000/TCP    15h
service/image-registry-operator   ClusterIP   None             <none>        60000/TCP   90d

NAME                     DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
daemonset.apps/node-ca   6         6         6       6            6           kubernetes.io/os=linux   90d

NAME                                              READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/cluster-image-registry-operator   1/1     1            1           90d
deployment.apps/image-registry                    1/1     1            1           15h

NAME                                                         DESIRED   CURRENT   READY   AGE
replicaset.apps/cluster-image-registry-operator-74f6d954b6   1         1         1       90d
replicaset.apps/image-registry-6758b547f                     1         1         1       76m
replicaset.apps/image-registry-78bfbd7f59                    0         0         0       15h
replicaset.apps/image-registry-7fcc8d6cc8                    0         0         0       80m
replicaset.apps/image-registry-864f88f5b                     0         0         0       15h
replicaset.apps/image-registry-cb47fffb                      0         0         0       10h

NAME                                COMPLETIONS   DURATION   AGE
job.batch/image-pruner-1627257600   1/1           10s        2d9h
job.batch/image-pruner-1627344000   1/1           6s         33h
job.batch/image-pruner-1627430400   1/1           5s         9h

NAME                         SCHEDULE    SUSPEND   ACTIVE   LAST SCHEDULE   AGE
cronjob.batch/image-pruner   0 0 * * *   False     0        9h              90d

NAME                                     HOST/PORT                                           PATH   SERVICES         PORT    TERMINATION   WILDCARD
route.route.openshift.io/public-routes   astra-registry.apps.ocp-vmw.cie.netapp.com          image-registry   <all>   reencrypt     None

If you are using the default TLS certificates for the ingress operator OpenShift registry route, you can fetch the TLS certificates using the following command.
```
[netapp-user@rhel7 ~]$ oc extract secret/router-ca --keys=tls.crt -n openshift-ingress-operator
```
To allow OpenShift nodes to access and pull the images from the registry, add the certificates to the docker client on the OpenShift nodes. Create a configmap in the openshift-config namespace using the TLS certificates and patch it to the cluster image config to make the certificate trusted.
```
[netapp-user@rhel7 ~]$ oc create configmap astra-ca -n openshift-config --from-file=astra-registry.apps.ocp-vmw.cie.netapp.com=tls.crt

[netapp-user@rhel7 ~]$ oc patch image.config.openshift.io/cluster --patch '{"spec":{"additionalTrustedCA":{"name":"astra-ca"}}}' --type=merge
```
The OpenShift internal registry is controlled by authentication. All the OpenShift users can access the OpenShift registry, but the operations that the logged in user can perform depends on the user permissions.
1. To allow a user or a group of users to pull images from the registry, the user(s) must have the registry-viewer role assigned.
  [netapp-user@rhel7 ~]$ oc policy add-role-to-user registry-viewer ocp-user [netapp-user@rhel7 ~]$ oc policy add-role-to-group registry-viewer ocp-user-group
2. To allow a user or group of users to write or push images, the user(s) must have the registry-editor role assigned.
  [netapp-user@rhel7 ~]$ oc policy add-role-to-user registry-editor ocp-user [netapp-user@rhel7 ~]$ oc policy add-role-to-group registry-editor ocp-user-group

For OpenShift nodes to access the registry and push or pull the images, you need to configure a pull secret.

[netapp-user@rhel7 ~]$ oc create secret docker-registry astra-registry-credentials --docker-server=astra-registry.apps.ocp-vmw.cie.netapp.com --docker-username=ocp-user --docker-password=password

This pull secret can then be patched to serviceaccounts or be referenced in the corresponding pod definition.
1. To patch it to service accounts, run the following command.
  [netapp-user@rhel7 ~]$ oc secrets link <service_account_name> astra-registry-credentials --for=pull
2. To reference the pull secret in the pod definition, add the following parameter to the spec section.
  imagePullSecrets: - name: astra-registry-credentials

To push or pull an image from workstations apart from OpenShift node, complete the following steps.

Add the TLS certificates to the docker client.

[netapp-user@rhel7 ~]$ sudo mkdir /etc/docker/certs.d/astra-registry.apps.ocp-vmw.cie.netapp.com

[netapp-user@rhel7 ~]$ sudo cp /path/to/tls.crt /etc/docker/certs.d/astra-registry.apps.ocp-vmw.cie.netapp.com

Log into OpenShift using the oc login command.

[netapp-user@rhel7 ~]$ oc login --token=sha256~D49SpB_lesSrJYwrM0LIO-VRcjWHu0a27vKa0 --server=https://api.ocp-vmw.cie.netapp.com:6443

Log into the registry using OpenShift user credentials with the podman/docker command.
podman
[netapp-user@rhel7 ~]$ podman login astra-registry.apps.ocp-vmw.cie.netapp.com -u kubeadmin -p $(oc whoami -t) --tls-verify=false

+
NOTE: If you are using kubeadmin user to log into the private registry, then use token instead of password.
docker
[netapp-user@rhel7 ~]$ docker login astra-registry.apps.ocp-vmw.cie.netapp.com -u kubeadmin -p $(oc whoami -t)

+
NOTE: If you are using kubeadmin user to log into the private registry, then use token instead of password.

Push or pull the images.

podman

[netapp-user@rhel7 ~]$ podman push astra-registry.apps.ocp-vmw.cie.netapp.com/netapp-astra/vault-controller:latest
[netapp-user@rhel7 ~]$ podman pull astra-registry.apps.ocp-vmw.cie.netapp.com/netapp-astra/vault-controller:latest

docker

[netapp-user@rhel7 ~]$ docker push astra-registry.apps.ocp-vmw.cie.netapp.com/netapp-astra/vault-controller:latest
[netapp-user@rhel7 ~]$ docker pull astra-registry.apps.ocp-vmw.cie.netapp.com/netapp-astra/vault-controller:latest

Creating Private Image Registries

Creating your file...

Creating A private image registry