Skip to main content
NetApp Solutions

Creating private image registries

Contributors ac-ntap
PDFs

For most deployments of Red Hat OpenShift, using a public registry like Quay.io or DockerHub meets most customer's needs. However there are times when a customer may want to host their own private or customized images.

This procedure documents creating a private image registry that is backed by a persistent volume provided by Astra Trident and NetApp ONTAP.

Note Astra Control Center requires a registry to host the images the Astra containers require. The following section describes the steps to setup a private registry on a Red Hat OpenShift cluster and pushing the images required to support the installation of Astra Control Center.

Creating A private image registry

  1. Remove the default annotation from the current default storage class and annotate the Trident-backed storage class as default for the OpenShift cluster.

    [netapp-user@rhel7 ~]$ oc patch storageclass thin -p '{"metadata": {"annotations": {"storageclass.kubernetes.io/is-default-class": "false"}}}'
storageclass.storage.k8s.io/thin patched

[netapp-user@rhel7 ~]$ oc patch storageclass ocp-trident -p '{"metadata": {"annotations": {"storageclass.kubernetes.io/is-default-class": "true"}}}'
storageclass.storage.k8s.io/ocp-trident patched

  2. Edit the imageregistry operator by entering the following storage parameters in the spec section.

    [netapp-user@rhel7 ~]$ oc edit configs.imageregistry.operator.openshift.io

storage:
  pvc:
    claim:

  3. Enter the following parameters in the spec section for creating a OpenShift route with a custom hostname. Save and exit.

    routes:
  - hostname: astra-registry.apps.ocp-vmw.cie.netapp.com
    name: netapp-astra-route
    Note The above route config is used when you want a custom hostname for your route. If you want OpenShift to create a route with a default hostname, you can add the following parameters to the spec section: defaultRoute: true.
    Custom TLS certificates

    When you are using a custom hostname for the route, by default, it uses the default TLS configuration of the OpenShift Ingress operator. However, you can add a custom TLS configuration to the route. To do so, complete the following steps.

    1. Create a secret with the route’s TLS certificates and key.

      [netapp-user@rhel7 ~]$ oc create secret tls astra-route-tls -n openshift-image-registry –cert/home/admin/netapp-astra/tls.crt --key=/home/admin/netapp-astra/tls.key

    2. Edit the imageregistry operator and add the following parameters to the spec section.

      [netapp-user@rhel7 ~]$ oc edit configs.imageregistry.operator.openshift.io

routes:
  - hostname: astra-registry.apps.ocp-vmw.cie.netapp.com
    name: netapp-astra-route
    secretName: astra-route-tls

  4. Edit the imageregistry operator again and change the management state of the operator to the Managed state. Save and exit.

    oc edit configs.imageregistry/cluster

managementState: Managed

  5. If all the prerequisites are satisfied, PVCs, pods, and services are created for the private image registry. In a few minutes, the registry should be up.

    [netapp-user@rhel7 ~]$oc get all -n openshift-image-registry

NAME                                                   READY   STATUS      RESTARTS   AGE
pod/cluster-image-registry-operator-74f6d954b6-rb7zr   1/1     Running     3          90d
pod/image-pruner-1627257600-f5cpj                      0/1     Completed   0          2d9h
pod/image-pruner-1627344000-swqx9                      0/1     Completed   0          33h
pod/image-pruner-1627430400-rv5nt                      0/1     Completed   0          9h
pod/image-registry-6758b547f-6pnj8                     1/1     Running     0          76m
pod/node-ca-bwb5r                                      1/1     Running     0          90d
pod/node-ca-f8w54                                      1/1     Running     0          90d
pod/node-ca-gjx7h                                      1/1     Running     0          90d
pod/node-ca-lcx4k                                      1/1     Running     0          33d
pod/node-ca-v7zmx                                      1/1     Running     0          7d21h
pod/node-ca-xpppp                                      1/1     Running     0          89d

NAME                              TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)     AGE
service/image-registry            ClusterIP   172.30.196.167   <none>        5000/TCP    15h
service/image-registry-operator   ClusterIP   None             <none>        60000/TCP   90d

NAME                     DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
daemonset.apps/node-ca   6         6         6       6            6           kubernetes.io/os=linux   90d

NAME                                              READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/cluster-image-registry-operator   1/1     1            1           90d
deployment.apps/image-registry                    1/1     1            1           15h

NAME                                                         DESIRED   CURRENT   READY   AGE
replicaset.apps/cluster-image-registry-operator-74f6d954b6   1         1         1       90d
replicaset.apps/image-registry-6758b547f                     1         1         1       76m
replicaset.apps/image-registry-78bfbd7f59                    0         0         0       15h
replicaset.apps/image-registry-7fcc8d6cc8                    0         0         0       80m
replicaset.apps/image-registry-864f88f5b                     0         0         0       15h
replicaset.apps/image-registry-cb47fffb                      0         0         0       10h

NAME                                COMPLETIONS   DURATION   AGE
job.batch/image-pruner-1627257600   1/1           10s        2d9h
job.batch/image-pruner-1627344000   1/1           6s         33h
job.batch/image-pruner-1627430400   1/1           5s         9h

NAME                         SCHEDULE    SUSPEND   ACTIVE   LAST SCHEDULE   AGE
cronjob.batch/image-pruner   0 0 * * *   False     0        9h              90d

NAME                                     HOST/PORT                                           PATH   SERVICES         PORT    TERMINATION   WILDCARD
route.route.openshift.io/public-routes   astra-registry.apps.ocp-vmw.cie.netapp.com          image-registry   <all>   reencrypt     None

  6. If you are using the default TLS certificates for the ingress operator OpenShift registry route, you can fetch the TLS certificates using the following command:

    [netapp-user@rhel7 ~]$ oc extract secret/router-ca --keys=tls.crt -n openshift-ingress-operator

  7. To allow OpenShift nodes to access and pull the images from the registry, add the certificates to the docker client on the OpenShift nodes. Create a configmap in the openshift-config namespace using the TLS certificates and patch it to the cluster image config to make the certificate trusted.

    [netapp-user@rhel7 ~]$ oc create configmap astra-ca -n openshift-config --from-file=astra-registry.apps.ocp-vmw.cie.netapp.com=tls.crt

[netapp-user@rhel7 ~]$ oc patch image.config.openshift.io/cluster --patch '{"spec":{"additionalTrustedCA":{"name":"astra-ca"}}}' --type=merge

  8. The OpenShift internal registry is controlled by authentication. All the OpenShift users can access the OpenShift registry, but the operations that the logged-in user can perform depends on the user permissions.

    1. To allow a user or a group of users to pull images from the registry, the user(s) must have the registry-viewer role assigned.

      [netapp-user@rhel7 ~]$ oc policy add-role-to-user registry-viewer ocp-user

[netapp-user@rhel7 ~]$ oc policy add-role-to-group registry-viewer ocp-user-group

    2. To allow a user or group of users to write or push images, the user(s) must have the registry-editor role assigned.

      [netapp-user@rhel7 ~]$ oc policy add-role-to-user registry-editor ocp-user

[netapp-user@rhel7 ~]$ oc policy add-role-to-group registry-editor ocp-user-group

  9. For OpenShift nodes to access the registry and push or pull the images, you need to configure a pull secret.

    [netapp-user@rhel7 ~]$ oc create secret docker-registry astra-registry-credentials --docker-server=astra-registry.apps.ocp-vmw.cie.netapp.com --docker-username=ocp-user --docker-password=password

  10. This pull secret can then be patched to serviceaccounts or be referenced in the corresponding pod definition.

    1. To patch it to service accounts, run the following command:

      [netapp-user@rhel7 ~]$ oc secrets link <service_account_name> astra-registry-credentials --for=pull

    2. To reference the pull secret in the pod definition, add the following parameter to the spec section.

      imagePullSecrets:
  - name: astra-registry-credentials

  11. To push or pull an image from workstations apart from OpenShift node, complete the following steps:

    1. Add the TLS certificates to the docker client.

      [netapp-user@rhel7 ~]$ sudo mkdir /etc/docker/certs.d/astra-registry.apps.ocp-vmw.cie.netapp.com

[netapp-user@rhel7 ~]$ sudo cp /path/to/tls.crt /etc/docker/certs.d/astra-registry.apps.ocp-vmw.cie.netapp.com

    2. Log into OpenShift using the oc login command.

      [netapp-user@rhel7 ~]$ oc login --token=sha256~D49SpB_lesSrJYwrM0LIO-VRcjWHu0a27vKa0 --server=https://api.ocp-vmw.cie.netapp.com:6443

    3. Log into the registry using OpenShift user credentials with the podman/docker command.

      podman
      [netapp-user@rhel7 ~]$ podman login astra-registry.apps.ocp-vmw.cie.netapp.com -u kubeadmin -p $(oc whoami -t) --tls-verify=false
      Note If you are using kubeadmin user to log into the private registry, then use a token instead of a password.
      docker
      [netapp-user@rhel7 ~]$ docker login astra-registry.apps.ocp-vmw.cie.netapp.com -u kubeadmin -p $(oc whoami -t)
      Note If you are using kubeadmin user to log into the private registry, then use a token instead of a password.

    4. Push or pull the images.

      podman
      [netapp-user@rhel7 ~]$ podman push astra-registry.apps.ocp-vmw.cie.netapp.com/netapp-astra/vault-controller:latest
[netapp-user@rhel7 ~]$ podman pull astra-registry.apps.ocp-vmw.cie.netapp.com/netapp-astra/vault-controller:latest
      docker
      [netapp-user@rhel7 ~]$ docker push astra-registry.apps.ocp-vmw.cie.netapp.com/netapp-astra/vault-controller:latest
[netapp-user@rhel7 ~]$ docker pull astra-registry.apps.ocp-vmw.cie.netapp.com/netapp-astra/vault-controller:latest

Next: Solution Validation/Use Cases.