Skip to main content

Configure a storage class

Contributors netapp-aruldeepa kcantrel

The Kubernetes StorageClass object identifies a provisioner and instructs the provisioner how to provision volumes. This section shows you how to configure a Kubernetes StorageClass object that specifies Trident as the provisioner.

Create a StorageClass Object

When you create a StorageClass for FSx for ONTAP, Trident will automatically create the backend configuration.

Note If you'd like to manually configure the storage backend, please refer to the Create a Kubernetes StorageClass without automatic backend configuration section for how to create the Trident backend and storage class separately.

Specify required StorageClass parameters

The following three parameters need to be defined when creating a StorageClass:

Parameter Required Type Description

fsxFilesystemID

Yes

string

FSx for NetApp ONTAP filesystem ID

storageDriverName

Yes

string

Trident storage driver (for example, ontap-nas or ontap-san)

credentialsName

Yes

string

Name of the Kubernetes Secret that contains FSx for ONTAP credentials

Specify optional parameters

You can pass optional backend parameters through the StorageClass. Define all optional values as strings in the StorageClass parameters section. For a complete list of backend parameters, see: FSx for NetApp ONTAP backend configuration.

Example StorageClass configuration files.

The following example shows a StorageClass that triggers automatic backend configuration.

YAML
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ontap-fsx-demo
  annotations:
    description: "Demo StorageClass for FSx for NetApp ONTAP"
provisioner: csi.trident.netapp.io
parameters:
  fsxFilesystemID: "fs-0abc123"
  storageDriverName: "ontap-nas"
  credentialsName: trident-fsx-credentials
allowVolumeExpansion: true
reclaimPolicy: Delete
volumeBindingMode: Immediate
JSON
{
  "apiVersion": "storage.k8s.io/v1",
  "kind": "StorageClass",
  "metadata": {
    "name": "ontap-fsx-demo",
    "annotations": {
      "description": "Demo StorageClass for FSx for NetApp ONTAP"
    }
  },
  "provisioner": "csi.trident.netapp.io",
  "parameters": {
    "fsxFilesystemID": "fs-0abc123",
    "storageDriverName": "ontap-nas",
    "credentialsName": "trident-fsx-credentials"
  },
  "allowVolumeExpansion": true,
  "reclaimPolicy": "Delete",
  "volumeBindingMode": "Immediate"
}

Create the StorageClass

Once you have created you configuration file, run the following command to create the storage class.

kubectl create -f storage-class-ontapnas.yaml

You should now see a basic-csi storage class in both Kubernetes and Trident, and Trident should have discovered the pools on the backend.

kubectl get sc basic-csi
NAME         PROVISIONER             AGE
basic-csi    csi.trident.netapp.io   15h

After you apply the StorageClass, Trident creates the backend automatically. You can then create PersistentVolumeClaims that reference this StorageClass.

Verify backend configuration status

Trident records the result of backend creation in StorageClass annotations.

Annotation Description

trident.netapp.io/configuratorStatus

Configuration result (Success or Failure)

trident.netapp.io/configuratorMessage

Detailed status or error message

trident.netapp.io/configuratorName

Name of the internal configurator resource

trident.netapp.io/managed

Indicates the StorageClass is managed by Trident

trident.netapp.io/additionalStoragePools

Storage pools created for this backend

To verify status, run:

kubectl get storageclass ontap-fsx-demo -o yaml

Confirm that trident.netapp.io/configuratorStatus is set to Success. If the value is Failure, review trident.netapp.io/configuratorMessage for the error.

Add additional FSxN file systems

If you need additional storage capacity while continuing to use the same StorageClass, add additional FSxN file system IDs.

Edit the StorageClass and add the following annotation:

metadata:
  annotations:
    trident.netapp.io/additionalFsxNFileSystemID: '["fs-xxxxxxxxxxxxxxxxxx"]'

After you apply the change, Trident updates the backend configuration and updates the StorageClass annotations.

Operational considerations and limitations

  • Deleting a StorageClass that as the automatic backend configuration usually deletes the associated Trident backend. This can disrupt storage connectivity and break running workloads. Validate the impact before you delete a managed StorageClass.

  • Automatic backend configuration is supported only for AWS FSx for NetApp ONTAP.

Create a Kubernetes StorageClass without automatic backend configuration

If you want to create the Trident backend and StorageClass separately then follow these steps.

Understand how automatic backend configuration works

Trident derives backend configuration from the StorageClass definition. When you apply the StorageClass, Trident validates the required parameters, creates the backend, and annotates the StorageClass with status.

Trident creates the VolumeSnapshotClass only once. Trident reuses the same VolumeSnapshotClass for subsequent StorageClasses.

Create the Trident backend

To create a Trident backend, you need to create a configuration file in either JSON or YAML format. The file needs to specify the type of storage you want (NAS or SAN), the file system, and SVM to get it from and how to authenticate with it. The following example shows how to define NAS-based storage and using an AWS secret to store the credentials to the SVM you want to use:

YAML
apiVersion: trident.netapp.io/v1
kind: TridentBackendConfig
metadata:
  name: backend-tbc-ontap-nas
  namespace: trident
spec:
  version: 1
  storageDriverName: ontap-nas
  backendName: tbc-ontap-nas
  svm: svm-name
  aws:
    fsxFilesystemID: fs-xxxxxxxxxx
  credentials:
    name: "arn:aws:secretsmanager:us-west-2:xxxxxxxx:secret:secret-name"
    type: awsarn
JSON
{
  "apiVersion": "trident.netapp.io/v1",
  "kind": "TridentBackendConfig",
  "metadata": {
    "name": "backend-tbc-ontap-nas"
    "namespace": "trident"
  },
  "spec": {
    "version": 1,
    "storageDriverName": "ontap-nas",
    "backendName": "tbc-ontap-nas",
    "svm": "svm-name",
    "aws": {
      "fsxFilesystemID": "fs-xxxxxxxxxx"
    },
    "managementLIF": null,
    "credentials": {
      "name": "arn:aws:secretsmanager:us-west-2:xxxxxxxx:secret:secret-name",
      "type": "awsarn"
    }
  }
}

FSx for ONTAP driver details

You can integrate Trident with Amazon FSx for NetApp ONTAP using the following drivers:

Driver Name Description

ontap-san

Each PV provisioned is a LUN within its own Amazon FSx for NetApp ONTAP volume. Recommended for block storage.

ontap-nas

Each PV provisioned is a full Amazon FSx for NetApp ONTAP volume. Recommended for NFS and SMB.

ontap-san-economy

Each PV provisioned is a LUN with a configurable number of LUNs per Amazon FSx for NetApp ONTAP volume.

ontap-nas-economy

Each PV provisioned is a qtree, with a configurable number of qtrees per Amazon FSx for NetApp ONTAP volume.

ontap-nas-flexgroup

Each PV provisioned is a full Amazon FSx for NetApp ONTAP FlexGroup volume.

For driver details, refer to NAS drivers and SAN drivers.

Create the backend

After creating the configuration file run the following commands to create and validate the Trident Backend Configuration (TBC):

  • Create trident backend configuration (TBC) from yaml file and run the following command:

    kubectl create -f backendconfig.yaml -n trident
    tridentbackendconfig.trident.netapp.io/backend-tbc-ontap-nas created
  • Validate the trident backend configuration (TBC) was created successfully:

    Kubectl get tbc -n trident
    NAME                         BACKEND NAME         BACKEND UUID                           PHASE   STATUS
    
    backend-tbc-ontap-nas        tbc-ontap-nas        933e0071-66ce-4324-b9ff-f96d916ac5e9   Bound   Success

For more information on other configuration options, see the [Backend-advanced-configuration-and-examples] section below.

Configure a Storage Class without automatic backend configuration

The following are examples of Storage Class configurations for use with Trident and FSx for ONTAP.

Storage Class for NFS

You can use this example to setup StorageClass for volumes using NFS (Refer to Trident Attribute section below for the full list of attributes):

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ontap-gold
provisioner: csi.trident.netapp.io
parameters:
  backendType: "ontap-nas"
  provisioningType: "thin"
  snapshots: "true"

Storage Calss for iSCSI

Use this example to setup StorageClass for volumes using iSCSI:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ontap-gold
provisioner: csi.trident.netapp.io
parameters:
  backendType: "ontap-san"
  provisioningType: "thin"
  snapshots: "true"

Storage Class using NFSv3 and AWS Bottlerocket

To provision NFSv3 volumes on AWS Bottlerocket, add the required mountOptions to the storage class:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ontap-gold
provisioner: csi.trident.netapp.io
parameters:
  backendType: "ontap-nas"
  media: "ssd"
  provisioningType: "thin"
  snapshots: "true"
mountOptions:
  - nfsvers=3
  - nolock

Trident StorageClass attributes

These parameters determine which Trident-managed storage pools should be utilized to provision volumes of a given type.

Attribute Type Values Offer Request Supported by

media1

string

hdd, hybrid, ssd

Pool contains media of this type; hybrid means both

Media type specified

ontap-nas, ontap-nas-economy, ontap-nas-flexgroup, ontap-san, solidfire-san

provisioningType

string

thin, thick

Pool supports this provisioning method

Provisioning method specified

thick: all ontap; thin: all ontap & solidfire-san

backendType

string

ontap-nas, ontap-nas-economy, ontap-nas-flexgroup, ontap-san, solidfire-san, azure-netapp-files, ontap-san-economy

Pool belongs to this type of backend

Backend specified

All drivers

snapshots

bool

true, false

Pool supports volumes with snapshots

Volume with snapshots enabled

ontap-nas, ontap-san, solidfire-san

clones

bool

true, false

Pool supports cloning volumes

Volume with clones enabled

ontap-nas, ontap-san, solidfire-san

encryption

bool

true, false

Pool supports encrypted volumes

Volume with encryption enabled

ontap-nas, ontap-nas-economy, ontap-nas-flexgroups, ontap-san

IOPS

int

positive integer

Pool is capable of guaranteeing IOPS in this range

Volume guaranteed these IOPS

solidfire-san

1: Not supported by ONTAP Select or FSx for ONTAP systems

Refer to Kubernetes and Trident objects for details on how storage classes interact with the PersistentVolumeClaim and parameters for controlling how Trident provisions volumes.

Create the storage class

Once you have configured the StorageClass, you can create it in Kubernetes.

Steps
  1. This is a Kubernetes object, so use kubectl to create it in Kubernetes.

    kubectl create -f storage-class-ontapnas.yaml
  2. You should now see a basic-csi storage class in both Kubernetes and Trident, and Trident should have discovered the pools on the backend.

    kubectl get sc basic-csi
NAME         PROVISIONER             AGE
basic-csi    csi.trident.netapp.io   15h

Provision SMB volumes

You can provision SMB volumes using the ontap-nas driver.
However, to do so you must complete these steps: Prepare to provision SMB volumes.

Backend advanced configuration and examples

See the following table for the backend configuration options:

Parameter Description Example

version

Always 1

storageDriverName

Name of the storage driver

ontap-nas, ontap-nas-economy, ontap-nas-flexgroup, ontap-san, ontap-san-economy

backendName

Custom name or the storage backend

Driver name + "_" + dataLIF

managementLIF

IP address of a cluster or SVM management LIF

A fully-qualified domain name (FQDN) can be specified.

Can be set to use IPv6 addresses if Trident was installed using the IPv6 flag. IPv6 addresses must be defined in square brackets, such as [28e8:d9fb:a825:b7bf:69a8:d02f:9e7b:3555].

If you provide the fsxFilesystemID under the aws field, you need not to provide the managementLIF because Trident retrieves the SVM managementLIF information from AWS. So, you must provide credentials for a user under the SVM (For example: vsadmin) and the user must have the vsadmin role.

"10.0.0.1", "[2001:1234:abcd::fefe]"

dataLIF

IP address of protocol LIF.

ONTAP NAS drivers: NetApp recommends specifying dataLIF. If not provided, Trident fetches dataLIFs from the SVM. You can specify a fully-qualified domain name (FQDN) to be used for the NFS mount operations, allowing you to create a round-robin DNS to load-balance across multiple dataLIFs. Can be changed after initial setting.

ONTAP SAN drivers: Do not specify for iSCSI. Trident uses ONTAP Selective LUN Map to discover the iSCI LIFs needed to establish a multi path session. A warning is generated if dataLIF is explicitly defined.

Can be set to use IPv6 addresses if Trident was installed using the IPv6 flag. IPv6 addresses must be defined in square brackets, such as [28e8:d9fb:a825:b7bf:69a8:d02f:9e7b:3555].

autoExportPolicy

Enable automatic export policy creation and updating [Boolean].

Using the autoExportPolicy and autoExportCIDRs options, Trident can manage export policies automatically.

false

autoExportCIDRs

List of CIDRs to filter Kubernetes' node IPs against when autoExportPolicy is enabled.

Using the autoExportPolicy and autoExportCIDRs options, Trident can manage export policies automatically.

"["0.0.0.0/0", "::/0"]"

labels

Set of arbitrary JSON-formatted labels to apply on volumes

""

clientCertificate

Base64-encoded value of client certificate. Used for certificate-based auth

""

clientPrivateKey

Base64-encoded value of client private key. Used for certificate-based auth

""

trustedCACertificate

Base64-encoded value of trusted CA certificate. Optional. Used for certificate-based authentication.

""

username

Username to connect to the cluster or SVM. Used for credential-based authentication. For example, vsadmin.

password

Password to connect to the cluster or SVM. Used for credential-based authentication.

svm

Storage virtual machine to use

Derived if an SVM managementLIF is specified.

storagePrefix

Prefix used when provisioning new volumes in the SVM.

Cannot be modified after creation. To update this parameter, you will need to create a new backend.

trident

limitAggregateUsage

Do not specify for Amazon FSx for NetApp ONTAP.

The provided fsxadmin and vsadmin do not contain the permissions required to retrieve aggregate usage and limit it using Trident.

Do not use.

limitVolumeSize

Fail provisioning if requested volume size is above this value.

Also restricts the maximum size of the volumes it manages for qtrees and LUNs, and the qtreesPerFlexvol option allows customizing the maximum number of qtrees per FlexVol volume

"" (not enforced by default)

lunsPerFlexvol

Maximum LUNs per Flexvol volume, must be in range [50, 200].

SAN only.

“100”

debugTraceFlags

Debug flags to use when troubleshooting. Example, {"api":false, "method":true}

Do not use debugTraceFlags unless you are troubleshooting and require a detailed log dump.

null

nfsMountOptions

Comma-separated list of NFS mount options.

The mount options for Kubernetes-persistent volumes are normally specified in storage classes, but if no mount options are specified in a storage class, Trident will fall back to using the mount options specified in the storage backend's configuration file.

If no mount options are specified in the storage class or the configuration file, Trident will not set any mount options on an associated persistent volume.

""

nasType

Configure NFS or SMB volumes creation.

Options are nfs, smb, or null.

Must set to smb for SMB volumes. Setting to null defaults to NFS volumes.

nfs

qtreesPerFlexvol

Maximum Qtrees per FlexVol volume, must be in range [50, 300]

"200"

smbShare

You can specify one of the following: the name of an SMB share created using the Microsoft Management Console or ONTAP CLI or a name to allow Trident to create the SMB share.

This parameter is required for Amazon FSx for ONTAP backends.

smb-share

useREST

Boolean parameter to use ONTAP REST APIs.

When set to true, Trident will use ONTAP REST APIs to communicate with the backend.

This feature requires ONTAP 9.11.1 and later. In addition, the ONTAP login role used must have access to the ontap application. This is satisfied by the pre-defined vsadmin and cluster-admin roles.

false

aws

You can specify the following in the configuration file for AWS FSx for ONTAP:
- fsxFilesystemID: Specify the ID of the AWS FSx file system.
- apiRegion: AWS API region name.
- apikey: AWS API key.
- secretKey: AWS secret key.





""
""
""

credentials

Specify the FSx SVM credentials to store in AWS Secrets Manager.
- name: Amazon Resource Name (ARN) of the secret, which contains the credentials of SVM.
- type: Set to awsarn.
Refer to Create an AWS Secrets Manager secret for more information.

Backend configuration options for provisioning volumes

You can control default provisioning using these options in the defaults section of the configuration. For an example, see the configuration examples below.

Parameter Description Default

spaceAllocation

Space-allocation for LUNs

true

spaceReserve

Space reservation mode; "none" (thin) or "volume" (thick)

none

snapshotPolicy

Snapshot policy to use

none

qosPolicy

QoS policy group to assign for volumes created. Choose one of qosPolicy or adaptiveQosPolicy per storage pool or backend.

Using QoS policy groups with Trident requires ONTAP 9.8 or later.

You should use a non-shared QoS policy group and ensuring the policy group is applied to each constituent individually. A shared QoS policy group enforces the ceiling for the total throughput of all workloads.

""

adaptiveQosPolicy

Adaptive QoS policy group to assign for volumes created. Choose one of qosPolicy or adaptiveQosPolicy per storage pool or backend.

Not supported by ontap-nas-economy.

""

snapshotReserve

Percentage of volume reserved for snapshots "0"

If snapshotPolicy is none, else ""

splitOnClone

Split a clone from its parent upon creation

false

encryption

Enable NetApp Volume Encryption (NVE) on the new volume; defaults to false. NVE must be licensed and enabled on the cluster to use this option.

If NAE is enabled on the backend, any volume provisioned in Trident will be NAE enabled.

For more information, refer to: How Trident works with NVE and NAE.

false

luksEncryption

Enable LUKS encryption. Refer to Use Linux Unified Key Setup (LUKS).

SAN only.

""

tieringPolicy

Tiering policy to use none

unixPermissions

Mode for new volumes.

Leave empty for SMB volumes.

""

securityStyle

Security style for new volumes.

NFS supports mixed and unix security styles.

SMB supports mixed and ntfs security styles.

NFS default is unix.

SMB default is ntfs.