Skip to main content

ONTAP SAN configuration options and examples

Contributors netapp-aruldeepa
PDFs

Learn how to create and use ONTAP SAN drivers with your Trident installation. This section provides backend configuration examples and details for mapping backends to StorageClasses.

Backend configuration options

See the following table for the backend configuration options:

Parameter Description Default

version

Always 1

storageDriverName

Name of the storage driver

ontap-nas, ontap-nas-economy, ontap-nas-flexgroup, ontap-san, ontap-san-economy

backendName

Custom name or the storage backend

Driver name + "_" + dataLIF

managementLIF

IP address of a cluster or SVM management LIF.

A fully-qualified domain name (FQDN) can be specified.

Can be set to use IPv6 addresses if Trident was installed using the IPv6 flag. IPv6 addresses must be defined in square brackets, such as [28e8:d9fb:a825:b7bf:69a8:d02f:9e7b:3555].

For seamless MetroCluster switchover, see the [mcc-best].

“10.0.0.1”, “[2001:1234:abcd::fefe]”

dataLIF

IP address of protocol LIF.

Do not specify for iSCSI. Trident uses ONTAP Selective LUN Map to discover the iSCI LIFs needed to establish a multi path session. A warning is generated if dataLIF is explicitly defined.

Omit for Metrocluster. See the [mcc-best].

Derived by the SVM

svm

Storage virtual machine to use

Omit for Metrocluster. See the [mcc-best].

Derived if an SVM managementLIF is specified

useCHAP

Use CHAP to authenticate iSCSI for ONTAP SAN drivers [Boolean].

Set to true for Trident to configure and use bidirectional CHAP as the default authentication for the SVM given in the backend. Refer to Prepare to configure backend with ONTAP SAN drivers for details.

false

chapInitiatorSecret

CHAP initiator secret. Required if useCHAP=true

""

labels

Set of arbitrary JSON-formatted labels to apply on volumes

""

chapTargetInitiatorSecret

CHAP target initiator secret. Required if useCHAP=true

""

chapUsername

Inbound username. Required if useCHAP=true

""

chapTargetUsername

Target username. Required if useCHAP=true

""

clientCertificate

Base64-encoded value of client certificate. Used for certificate-based auth

""

clientPrivateKey

Base64-encoded value of client private key. Used for certificate-based auth

""

trustedCACertificate

Base64-encoded value of trusted CA certificate. Optional. Used for certificate-based authentication.

""

username

Username needed to communicate with the ONTAP cluster. Used for credential-based authentication.

""

password

Password needed to communicate with the ONTAP cluster. Used for credential-based authentication.

""

svm

Storage virtual machine to use

Derived if an SVM managementLIF is specified

storagePrefix

Prefix used when provisioning new volumes in the SVM.

Cannot be modified later. To update this parameter, you will need to create a new backend.

trident

aggregate

Aggregate for provisioning (optional; if set, must be assigned to the SVM). For the ontap-nas-flexgroup driver, this option is ignored. If not assigned, any of the available aggregates can be used to provision a FlexGroup volume.

Note When the aggregate is updated in SVM, it is updated in Trident automatically by polling SVM without having to restart the Trident Controller. When you have configured a specific aggregate in Trident to provision volumes, if the aggregate is renamed or moved out of the SVM, the backend will move to failed state in Trident while polling the SVM aggregate. You must either change the aggregate to one that is present on the SVM or remove it altogether to bring the backend back online.

""

limitAggregateUsage

Fail provisioning if usage is above this percentage.

If you are using an Amazon FSx for NetApp ONTAP backend, do not specify limitAggregateUsage. The provided fsxadmin and vsadmin do not contain the permissions required to retrieve aggregate usage and limit it using Trident.

"" (not enforced by default)

limitVolumeSize

Fail provisioning if requested volume size is above this value.

Also restricts the maximum size of the volumes it manages for LUNs.

"" (not enforced by default)

lunsPerFlexvol

Maximum LUNs per Flexvol, must be in range [50, 200]

100

debugTraceFlags

Debug flags to use when troubleshooting. Example, {"api":false, "method":true}

Do not use unless you are troubleshooting and require a detailed log dump.

null

useREST

Boolean parameter to use ONTAP REST APIs.

useREST When set to true, Trident uses ONTAP REST APIs to communicate with the backend; when set to false, Trident uses ONTAP ZAPI calls to communicate with the backend. This feature requires ONTAP 9.11.1 and later. In addition, the ONTAP login role used must have access to the ontap application. This is satisfied by the pre-defined vsadmin and cluster-admin roles. Beginning with the Trident 24.06 release and ONTAP 9.15.1 or later, userREST is set to true by default; change useREST to false to use ONTAP ZAPI calls.

useREST is fully qualified for NVMe/TCP.

true for ONTAP 9.15.1 or later, otherwise false.

sanType

Use to select iscsi for iSCSI, nvme for NVMe/TCP or fcp for SCSI over Fibre Channel (FC).

'fcp' (SCSI over FC) is a tech preview feature in the Trident 24.10 release.

iscsi if blank

formatOptions

Use formatOptions to specify command line arguments for the mkfs command, which will be applied whenever a volume is formatted. This allows you to format the volume according to your preferences. Make sure to specify the formatOptions similar to that of the mkfs command options, excluding the device path.
Example: "-E nodiscard"

Supported for ontap-san and ontap-san-economy drivers only.

limitVolumePoolSize

Maximum requestable FlexVol size when using LUNs in ontap-san-economy backend.

"" (not enforced by default)

denyNewVolumePools

Restricts ontap-san-economy backends from creating new FlexVol volumes to contain their LUNs. Only preexisting Flexvols are used for provisioning new PVs.

Recommendations for using formatOptions

Trident recommends the following option to expedite the formatting process:

-E nodiscard:

  • Keep, do not attempt to discard blocks at mkfs time (discarding blocks initially is useful on solid state devices and sparse / thin-provisioned storage). This replaces the deprecated option “-K” and it is applicable to all the file systems (xfs, ext3, and ext4).

Backend configuration options for provisioning volumes

You can control default provisioning using these options in the defaults section of the configuration. For an example, see the configuration examples below.

Parameter Description Default

spaceAllocation

Space-allocation for LUNs

"true"

spaceReserve

Space reservation mode; "none" (thin) or "volume" (thick)

"none"

snapshotPolicy

Snapshot policy to use

"none"

qosPolicy

QoS policy group to assign for volumes created. Choose one of qosPolicy or adaptiveQosPolicy per storage pool/backend.

Using QoS policy groups with Trident requires ONTAP 9.8 or later. You should use a non-shared QoS policy group and ensuring the policy group is applied to each constituent individually. A shared QoS policy group enforces the ceiling for the total throughput of all workloads.

""

adaptiveQosPolicy

Adaptive QoS policy group to assign for volumes created. Choose one of qosPolicy or adaptiveQosPolicy per storage pool/backend

""

snapshotReserve

Percentage of volume reserved for snapshots

"0" if snapshotPolicy is "none", otherwise ""

splitOnClone

Split a clone from its parent upon creation

"false"

encryption

Enable NetApp Volume Encryption (NVE) on the new volume; defaults to false. NVE must be licensed and enabled on the cluster to use this option.

If NAE is enabled on the backend, any volume provisioned in Trident will be NAE enabled.

For more information, refer to: How Trident works with NVE and NAE.

"false"

luksEncryption

Enable LUKS encryption. Refer to Use Linux Unified Key Setup (LUKS).

LUKS encryption is not supported for NVMe/TCP.

""

securityStyle

Security style for new volumes

unix

tieringPolicy

Tiering policy to use "none"

"snapshot-only" for pre-ONTAP 9.5 SVM-DR configuration

nameTemplate

Template to create custom volume names.

""

Volume provisioning examples

Here's an example with defaults defined:

---
version: 1
storageDriverName: ontap-san
managementLIF: 10.0.0.1
svm: trident_svm
username: admin
password: <password>
labels:
  k8scluster: dev2
  backend: dev2-sanbackend
storagePrefix: alternate-trident
debugTraceFlags:
  api: false
  method: true
defaults:
  spaceReserve: volume
  qosPolicy: standard
  spaceAllocation: 'false'
  snapshotPolicy: default
  snapshotReserve: '10'
Note For all volumes created using the ontap-san driver, Trident adds an extra 10 percent capacity to the FlexVol to accommodate the LUN metadata. The LUN will be provisioned with the exact size that the user requests in the PVC. Trident adds 10 percent to the FlexVol (shows as Available size in ONTAP). Users will now get the amount of usable capacity they requested. This change also prevents LUNs from becoming read-only unless the available space is fully utilized. This does not apply to ontap-san-economy.

For backends that define snapshotReserve, Trident calculates the size of volumes as follows:

Total volume size = [(PVC requested size) / (1 - (snapshotReserve percentage) / 100)] * 1.1

The 1.1 is the extra 10 percent Trident adds to the FlexVol to accommodate the LUN metadata. For snapshotReserve = 5%, and PVC request = 5GiB, the total volume size is 5.79GiB and the available size is 5.5GiB. The volume show command should show results similar to this example:

Shows the output of the volume show command.

Currently, resizing is the only way to use the new calculation for an existing volume.

Minimal configuration examples

The following examples show basic configurations that leave most parameters to default. This is the easiest way to define a backend.

Note If you are using Amazon FSx on NetApp ONTAP with Trident, we recommend you specify DNS names for LIFs instead of IP addresses.
ONTAP SAN example

This is a basic configuration using the ontap-san driver.

---
version: 1
storageDriverName: ontap-san
managementLIF: 10.0.0.1
svm: svm_iscsi
labels:
  k8scluster: test-cluster-1
  backend: testcluster1-sanbackend
username: vsadmin
password: <password>
ONTAP SAN economy example 
---
version: 1
storageDriverName: ontap-san-economy
managementLIF: 10.0.0.1
svm: svm_iscsi_eco
username: vsadmin
password: <password>

  1. example

You can configure the backend to avoid having to manually update the backend definition after switchover and switchback during SVM replication and recovery.

For seamless switchover and switchback, specify the SVM using managementLIF and omit the dataLIF and svm parameters. For example:

---
version: 1
storageDriverName: ontap-san
managementLIF: 192.168.1.66
username: vsadmin
password: password
Certificate-based authentication example

In this basic configuration example clientCertificate, clientPrivateKey, and trustedCACertificate (optional, if using trusted CA) are populated in backend.json and take the base64-encoded values of the client certificate, private key, and trusted CA certificate, respectively.

---
version: 1
storageDriverName: ontap-san
backendName: DefaultSANBackend
managementLIF: 10.0.0.1
svm: svm_iscsi
useCHAP: true
chapInitiatorSecret: cl9qxIm36DKyawxy
chapTargetInitiatorSecret: rqxigXgkesIpwxyz
chapTargetUsername: iJF4heBRT0TCwxyz
chapUsername: uh2aNCLSd6cNwxyz
clientCertificate: ZXR0ZXJwYXB...ICMgJ3BhcGVyc2
clientPrivateKey: vciwKIyAgZG...0cnksIGRlc2NyaX
trustedCACertificate: zcyBbaG...b3Igb3duIGNsYXNz
Bidirectional CHAP examples

These examples create a backend with useCHAP set to true.

ONTAP SAN CHAP example
---
version: 1
storageDriverName: ontap-san
managementLIF: 10.0.0.1
svm: svm_iscsi
labels:
  k8scluster: test-cluster-1
  backend: testcluster1-sanbackend
useCHAP: true
chapInitiatorSecret: cl9qxIm36DKyawxy
chapTargetInitiatorSecret: rqxigXgkesIpwxyz
chapTargetUsername: iJF4heBRT0TCwxyz
chapUsername: uh2aNCLSd6cNwxyz
username: vsadmin
password: <password>
ONTAP SAN economy CHAP example
---
version: 1
storageDriverName: ontap-san-economy
managementLIF: 10.0.0.1
svm: svm_iscsi_eco
useCHAP: true
chapInitiatorSecret: cl9qxIm36DKyawxy
chapTargetInitiatorSecret: rqxigXgkesIpwxyz
chapTargetUsername: iJF4heBRT0TCwxyz
chapUsername: uh2aNCLSd6cNwxyz
username: vsadmin
password: <password>
NVMe/TCP example

You must have an SVM configured with NVMe on your ONTAP backend. This is a basic backend configuration for NVMe/TCP.

---
version: 1
backendName: NVMeBackend
storageDriverName: ontap-san
managementLIF: 10.0.0.1
svm: svm_nvme
username: vsadmin
password: password
sanType: nvme
useREST: true
Backend configuration example with nameTemplate 
---
version: 1
storageDriverName: ontap-san
backendName: ontap-san-backend
managementLIF: <ip address>
svm: svm0
username: <admin>
password: <password>
defaults: {
    "nameTemplate": "{{.volume.Name}}_{{.labels.cluster}}_{{.volume.Namespace}}_{{.volume.RequestName}}"
},
"labels": {"cluster": "ClusterA", "PVC": "{{.volume.Namespace}}_{{.volume.RequestName}}"}
formatOptions example for ontap-san-economy driver 
version: 1
storageDriverName: ontap-san-economy
managementLIF: ''
svm: svm1
username: ''
password: "!"
storagePrefix: whelk_
debugTraceFlags:
  method: true
  api: true
defaults:
  formatOptions: "-E nodiscard"

Examples of backends with virtual pools

In these sample backend definition files, specific defaults are set for all storage pools, such as spaceReserve at none, spaceAllocation at false, and encryption at false. The virtual pools are defined in the storage section.

Trident sets provisioning labels in the "Comments" field. Comments are set on the FlexVol. Trident copies all labels present on a virtual pool to the storage volume at provisioning. For convenience, storage administrators can define labels per virtual pool and group volumes by label.

In these examples, some of the storage pools set their own spaceReserve, spaceAllocation, and encryption values, and some pools override the default values.

ONTAP SAN example 
---
version: 1
storageDriverName: ontap-san
managementLIF: 10.0.0.1
svm: svm_iscsi
useCHAP: true
chapInitiatorSecret: cl9qxIm36DKyawxy
chapTargetInitiatorSecret: rqxigXgkesIpwxyz
chapTargetUsername: iJF4heBRT0TCwxyz
chapUsername: uh2aNCLSd6cNwxyz
username: vsadmin
password: <password>
defaults:
  spaceAllocation: 'false'
  encryption: 'false'
  qosPolicy: standard
labels:
  store: san_store
  kubernetes-cluster: prod-cluster-1
region: us_east_1
storage:
- labels:
    protection: gold
    creditpoints: '40000'
  zone: us_east_1a
  defaults:
    spaceAllocation: 'true'
    encryption: 'true'
    adaptiveQosPolicy: adaptive-extreme
- labels:
    protection: silver
    creditpoints: '20000'
  zone: us_east_1b
  defaults:
    spaceAllocation: 'false'
    encryption: 'true'
    qosPolicy: premium
- labels:
    protection: bronze
    creditpoints: '5000'
  zone: us_east_1c
  defaults:
    spaceAllocation: 'true'
    encryption: 'false'
ONTAP SAN economy example 
---
version: 1
storageDriverName: ontap-san-economy
managementLIF: 10.0.0.1
svm: svm_iscsi_eco
useCHAP: true
chapInitiatorSecret: cl9qxIm36DKyawxy
chapTargetInitiatorSecret: rqxigXgkesIpwxyz
chapTargetUsername: iJF4heBRT0TCwxyz
chapUsername: uh2aNCLSd6cNwxyz
username: vsadmin
password: <password>
defaults:
  spaceAllocation: 'false'
  encryption: 'false'
labels:
  store: san_economy_store
region: us_east_1
storage:
- labels:
    app: oracledb
    cost: '30'
  zone: us_east_1a
  defaults:
    spaceAllocation: 'true'
    encryption: 'true'
- labels:
    app: postgresdb
    cost: '20'
  zone: us_east_1b
  defaults:
    spaceAllocation: 'false'
    encryption: 'true'
- labels:
    app: mysqldb
    cost: '10'
  zone: us_east_1c
  defaults:
    spaceAllocation: 'true'
    encryption: 'false'
- labels:
    department: legal
    creditpoints: '5000'
  zone: us_east_1c
  defaults:
    spaceAllocation: 'true'
    encryption: 'false'
NVMe/TCP example 
---
version: 1
storageDriverName: ontap-san
sanType: nvme
managementLIF: 10.0.0.1
svm: nvme_svm
username: vsadmin
password: <password>
useREST: true
defaults:
  spaceAllocation: 'false'
  encryption: 'true'
storage:
- labels:
    app: testApp
    cost: '20'
  defaults:
    spaceAllocation: 'false'
    encryption: 'false'

Map backends to StorageClasses

The following StorageClass definitions refer to the Examples of backends with virtual pools. Using the parameters.selector field, each StorageClass calls out which virtual pools can be used to host a volume. The volume will have the aspects defined in the chosen virtual pool.

  • The protection-gold StorageClass will map to the first virtual pool in the ontap-san backend. This is the only pool offering gold-level protection.

    apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: protection-gold
provisioner: csi.trident.netapp.io
parameters:
  selector: "protection=gold"
  fsType: "ext4"

  • The protection-not-gold StorageClass will map to the second and third virtual pool in ontap-san backend. These are the only pools offering a protection level other than gold.

    apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: protection-not-gold
provisioner: csi.trident.netapp.io
parameters:
  selector: "protection!=gold"
  fsType: "ext4"

  • The app-mysqldb StorageClass will map to the third virtual pool in ontap-san-economy backend. This is the only pool offering storage pool configuration for the mysqldb type app.

    apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: app-mysqldb
provisioner: csi.trident.netapp.io
parameters:
  selector: "app=mysqldb"
  fsType: "ext4"

  • The protection-silver-creditpoints-20k StorageClass will map to the second virtual pool in ontap-san backend. This is the only pool offering silver-level protection and 20000 creditpoints.

    apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: protection-silver-creditpoints-20k
provisioner: csi.trident.netapp.io
parameters:
  selector: "protection=silver; creditpoints=20000"
  fsType: "ext4"

  • The creditpoints-5k StorageClass will map to the third virtual pool in ontap-san backend and the fourth virtual pool in the ontap-san-economy backend. These are the only pool offerings with 5000 creditpoints.

    apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: creditpoints-5k
provisioner: csi.trident.netapp.io
parameters:
  selector: "creditpoints=5000"
  fsType: "ext4"

  • The my-test-app-sc StorageClass will map to the testAPP virtual pool in the ontap-san driver with sanType: nvme. This is the only pool offering testApp.

    ---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: my-test-app-sc
provisioner: csi.trident.netapp.io
parameters:
  selector: "app=testApp"
  fsType: "ext4"

Trident will decide which virtual pool is selected and ensures the storage requirement is met.