Skip to main content
NetApp Solutions

Red Hat OpenShift Service on AWS with NetApp ONTAP

Contributors banum-netapp
PDFs

Overview

In this section, we will show how to utilize FSx for ONTAP as a persistent storage layer for applications running on ROSA. It will show the installation of the NetApp Trident CSI driver on a ROSA cluster, the provisioning of an FSx for ONTAP file system, and the deployment of a sample stateful application. It will also show strategies for backing up and restoring your application data. With this integrated solution, you can establish a shared storage framework that effortlessly scales across AZs, simplifying the processes of scaling, protecting, and restoring your data using the Trident CSI driver.

Prerequisites

This diagram shows the ROSA cluster deployed in multiple AZs. ROSA cluster’s master nodes, infrastructure nodes are in Red Hat’s VPC, while the worker nodes are in a VPC in the customer's account . We’ll create an FSx for ONTAP file system within the same VPC and install the Trident driver in the ROSA cluster, allowing all the subnets of this VPC to connect to the file system.

Rosa architecture

Initial Setup

1. Provision FSx for NetApp ONTAP

Create a multi-AZ FSx for NetApp ONTAP in the same VPC as the ROSA cluster. There are several ways to do this. The details of creating FSxN using a CloudFormation Stack are provided

a.Clone the GitHub repository

$ git clone https://github.com/aws-samples/rosa-fsx-netapp-ontap.git

b.Run the CloudFormation Stack
Run the command below by replacing the parameter values with your own values:

$ cd rosa-fsx-netapp-ontap/fsx
$ aws cloudformation create-stack \
  --stack-name ROSA-FSXONTAP \
  --template-body file://./FSxONTAP.yaml \
  --region <region-name> \
  --parameters \
  ParameterKey=Subnet1ID,ParameterValue=[subnet1_ID] \
  ParameterKey=Subnet2ID,ParameterValue=[subnet2_ID] \
  ParameterKey=myVpc,ParameterValue=[VPC_ID] \
ParameterKey=FSxONTAPRouteTable,ParameterValue=[routetable1_ID,routetable2_ID] \
  ParameterKey=FileSystemName,ParameterValue=ROSA-myFSxONTAP \
  ParameterKey=ThroughputCapacity,ParameterValue=1024 \
  ParameterKey=FSxAllowedCIDR,ParameterValue=[your_allowed_CIDR] \
  ParameterKey=FsxAdminPassword,ParameterValue=[Define Admin password] \
  ParameterKey=SvmAdminPassword,ParameterValue=[Define SVM password] \
  --capabilities CAPABILITY_NAMED_IAM

Where :
region-name: same as the region where the ROSA cluster is deployed
subnet1_ID : id of the Preferred subnet for FSxN
subnet2_ID: id of the Standby subnet for FSxN
VPC_ID: id of the VPC where the ROSA cluster is deployed
routetable1_ID, routetable2_ID: ids of the route tables associated with the subnets chosen above
your_allowed_CIDR: allowed CIDR range for the FSx for ONTAP security groups ingress rules to
control access. You can use 0.0.0.0/0 or any appropriate CIDR to allow all
traffic to access the specific ports of FSx for ONTAP.
Define Admin password: A password to login to FSxN
Define SVM password: A password to login to SVM that will be created.

Verify that your file system and storage virtual machine (SVM) has been created using the Amazon FSx console, shown below:

FSxN created

2.Install and configure Trident CSI driver for the ROSA cluster

a.Add the Trident Helm repository

$ helm repo add netapp-trident https://netapp.github.io/trident-helm-chart

b.Install trident using helm

$ helm install trident netapp-trident/trident-operator --version 100.2406.0 --create-namespace --namespace trident
Note Depending on the version you install, the version parameter will need to be changed in the command shown. Refer to the documentation for the correct version number. For additional methods of installing Trident, refer to the Trident documentation.

c.Verify that all Trident pods are in the running state

Trident pods running

3. Configure the Trident CSI backend to use FSx for ONTAP (ONTAP NAS)

The Trident back-end configuration tells Trident how to communicate with the storage system (in this case, FSx for ONTAP). For creating the backend, we will provide the credentials of the Storage Virtual machine to connect to, along with the Cluster Management and the NFS data interfaces. We will use the ontap-nas driver to provision storage volumes in FSx file system.

a. First, create a secret for the SVM credentials using the following yaml

apiVersion: v1
kind: Secret
metadata:
  name: backend-fsx-ontap-nas-secret
  namespace: trident
type: Opaque
stringData:
  username: vsadmin
  password: <value provided for Define SVM password as a parameter to the Cloud Formation Stack>
Note You can also retrieve the SVM password created for FSxN from the AWS Secrets Manager as shown below.

AWS secrets Manager

retrieve secret

b.Next, add the secret for the SVM credentials to the ROSA cluster using the following command

$ oc apply -f svm_secret.yaml

You can verify that the secret has been added in the trident namespace using the following command

$ oc get secrets -n trident |grep backend-fsx-ontap-nas-secret

secret applied

c. Next, create the backend object
For this, move into the fsx directory of your cloned Git repository. Open the file backend-ontap-nas.yaml. Replace the following:
managementLIF with the Management DNS name
dataLIF with the NFS DNS name of the Amazon FSx SVM and
svm with the SVM name. Create the backend object using the following command.

Create the backend object using the following command.

$ oc apply -f backend-ontap-nas.yaml
Note You can get the Management DNS name, NFS DNS name and the SVM name from the Amazon FSx Console as shown in the screenshot below

get lifs

d. Now, run the following command to verify that the backend object has been created and Phase is showing Bound and Status is Success

create backend

4. Create Storage Class
Now that the Trident backend is configured, you can create a Kubernetes storage class to use the backend. Storage class is a resource object made available to the cluster. It describes and classifies the type of storage that you can request for an application.

a. Review the file storage-class-csi-nas.yaml in the fsx folder.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: trident-csi
provisioner: csi.trident.netapp.io
parameters:
  backendType: "ontap-nas"
  fsType: "ext4"
allowVolumeExpansion: True
reclaimPolicy: Retain

b. Create Storage Class in ROSA cluster and verify that trident-csi storage class has been created.

create backend

This completes the installation of Trident CSI driver and its connectivity to FSx for ONTAP file system. Now you can deploy a sample Postgresql stateful application on ROSA using file volumes on FSx for ONTAP.

c. Verify that there are no PVCs and PVs created using the trident-csi storage class.

no PVCs using Trident

d. Verify that applications can create PV using Trident CSI.

Create a PVC using the pvc-trident.yaml file provided in the fsx folder.

pvc-trident.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: basic
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 10Gi
  storageClassName: trident-csi
You can issue the following commands to create a pvc and verify that it has been created.

create test PVC using Trident

5. Deploy a sample Postgresql stateful application

a. Use helm to install postgresql

$ helm install postgresql bitnami/postgresql -n postgresql --create-namespace

install postgresql

b. Verify that the application pod is running, and a PVC and PV is created for the application.

postgresql pods

postgresql pvc

postgresql pv

c. Deploy a Postgresql client

Use the following command to get the password for the postgresql server that was installed.

$ export POSTGRES_PASSWORD=$(kubectl get secret --namespace postgresql postgresql -o jsoata.postgres-password}" | base64 -d)

Use the following command to run a postgresql client and connect to the server using the password

$ kubectl run postgresql-client --rm --tty -i --restart='Never' --namespace postgresql --image docker.io/bitnami/postgresql:16.2.0-debian-11-r1 --env="PGPASSWORD=$POSTGRES_PASSWORD" \
> --command -- psql --host postgresql -U postgres -d postgres -p 5432

postgresql client

d. Create a database and a table. Create a schema for the table and insert 2 rows of data into the table.

postgresql table

postgresql row1

postgresql rows2