Customize Trident operator installation

10/31/2024 Contributors

The Trident operator allows you to customize Trident installation using the attributes in the TridentOrchestrator spec. If you want to customize the installation beyond what TridentOrchestrator arguments allow, consider using tridentctl to generate custom YAML manifests to modify as needed.

Understanding controller pods and node pods

Trident runs as a single controller pod, plus a node pod on each worker node in the cluster. The node pod must be running on any host where you want to potentially mount a Trident volume.

Kubernetes node selectors and tolerations and taints are used to constrain a pod to run on a specific or preferred node. Using the`ControllerPlugin` and NodePlugin, you can specify constraints and overrides.

The controller plugin handles volume provisioning and management, such as snapshots and resizing.
The node plugin handles attaching the storage to the node.

Configuration options

spec.namespace is specified in TridentOrchestrator to signify the namespace where Trident is installed. This parameter cannot be updated after Trident is installed. Attempting to do so causes the TridentOrchestrator status to change to Failed. Trident is not intended to be migrated across namespaces.

This table details TridentOrchestrator attributes.

Parameter Description Default

Parameter	Description	Default
`namespace`	Namespace to install Trident in	`"default"`
`debug`	Enable debugging for Trident	`false`
`enableForceDetach`	`ontap-san`, `ontap-san-economy`, and `ontap-nas-economy` only. Works with Kubernetes Non-Graceful Node Shutdown (NGNS) to grant cluster administrators ability to safely migrate workloads with mounted volumes to new nodes should a node become unhealthy.	`false`
`windows`	Setting to `true` enables installation on Windows worker nodes.	`false`
`cloudProvider`	Set to `"Azure"` when using managed identities or a cloud identity on an AKS cluster. Set to "AWS" when using a cloud identity on an EKS cluster.	`""`
`cloudIdentity`	Set to workload identity ("azure.workload.identity/client-id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx") when using cloud identity on an AKS cluster. Set to AWS IAM role ("'eks.amazonaws.com/role-arn: arn:aws:iam::123456:role/trident-role") when using cloud identity on an EKS cluster.	`""`
`IPv6`	Install Trident over IPv6	false
`k8sTimeout`	Timeout for Kubernetes operations	`30sec`
`silenceAutosupport`	Don't send autosupport bundles to NetApp automatically	`false`
`autosupportImage`	The container image for Autosupport Telemetry	`"netapp/trident-autosupport:24.10"`
`autosupportProxy`	The address/port of a proxy for sending Autosupport Telemetry	`"http://proxy.example.com:8888"`
`uninstall`	A flag used to uninstall Trident	`false`
`logFormat`	Trident logging format to be used [text,json]	`"text"`
`tridentImage`	Trident image to install	`"netapp/trident:24.10"`
`imageRegistry`	Path to internal registry, of the format `<registry FQDN>[:port][/subpath]`	`"k8s.gcr.io"` (Kubernetes 1.19+) or `"quay.io/k8scsi"`
`kubeletDir`	Path to the kubelet directory on the host	`"/var/lib/kubelet"`
`wipeout`	A list of resources to delete to perform a complete removal of Trident
`imagePullSecrets`	Secrets to pull images from an internal registry
`imagePullPolicy`	Sets the image pull policy for the the Trident operator. Valid values are: `Always` to always pull the image. `IfNotPresent` to pull the image only if it does not already exist on the node. `Never` to never pull the image.	`IfNotPresent`
`controllerPluginNodeSelector`	Additional node selectors for pods. Follows same format as `pod.spec.nodeSelector`.	No default; optional
`controllerPluginTolerations`	Overrides Kubernetes tolerations for pods. Follows the same format as `pod.spec.Tolerations`.	No default; optional
`nodePluginNodeSelector`	Additional node selectors for pods. Follows same format as `pod.spec.nodeSelector`.	No default; optional
`nodePluginTolerations`	Overrides Kubernetes tolerations for pods. Follows the same format as `pod.spec.Tolerations`.	No default; optional
`nodePrep`	Enables Trident to prepare the nodes of the Kubernetes cluster to manage volumes using the specified data storage protocol. Currently, `iscsi` is the only value supported.

namespace

Namespace to install Trident in

"default"

debug

Enable debugging for Trident

false

enableForceDetach

ontap-san, ontap-san-economy, and ontap-nas-economy only.

Works with Kubernetes Non-Graceful Node Shutdown (NGNS) to grant cluster administrators ability to safely migrate workloads with mounted volumes to new nodes should a node become unhealthy.

false

windows

Setting to true enables installation on Windows worker nodes.

false

cloudProvider

Set to "Azure" when using managed identities or a cloud identity on an AKS cluster. Set to "AWS" when using a cloud identity on an EKS cluster.

""

cloudIdentity

Set to workload identity ("azure.workload.identity/client-id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx") when using cloud identity on an AKS cluster. Set to AWS IAM role ("'eks.amazonaws.com/role-arn: arn:aws:iam::123456:role/trident-role") when using cloud identity on an EKS cluster.

""

IPv6

Install Trident over IPv6

false

k8sTimeout

Timeout for Kubernetes operations

30sec

silenceAutosupport

Don't send autosupport bundles to NetApp
automatically

false

autosupportImage

The container image for Autosupport Telemetry

"netapp/trident-autosupport:24.10"

autosupportProxy

The address/port of a proxy for sending Autosupport
Telemetry

"http://proxy.example.com:8888"

uninstall

A flag used to uninstall Trident

false

logFormat

Trident logging format to be used [text,json]

"text"

tridentImage

Trident image to install

"netapp/trident:24.10"

imageRegistry

Path to internal registry, of the format
<registry FQDN>[:port][/subpath]

"k8s.gcr.io" (Kubernetes 1.19+)
or "quay.io/k8scsi"

kubeletDir

Path to the kubelet directory on the host

"/var/lib/kubelet"

wipeout

A list of resources to delete to perform a complete removal of
Trident

imagePullSecrets

Secrets to pull images from an internal registry

imagePullPolicy

Sets the image pull policy for the the Trident operator. Valid values are:

Always to always pull the image.

IfNotPresent to pull the image only if it does not already exist on the node.

Never to never pull the image.

IfNotPresent

controllerPluginNodeSelector

Additional node selectors for pods. Follows same format as pod.spec.nodeSelector.

No default; optional

controllerPluginTolerations

Overrides Kubernetes tolerations for pods. Follows the same format as pod.spec.Tolerations.

No default; optional

nodePluginNodeSelector

Additional node selectors for pods. Follows same format as pod.spec.nodeSelector.

No default; optional

nodePluginTolerations

Overrides Kubernetes tolerations for pods. Follows the same format as pod.spec.Tolerations.

No default; optional

nodePrep

Enables Trident to prepare the nodes of the Kubernetes cluster to manage volumes using the specified data storage protocol.
Currently, iscsi is the only value supported.

For more information on formatting pod parameters, refer to Assigning Pods to Nodes.

Details about force detach

Force detach is available for ontap-san, ontap-san-economy and onatp-nas-economy only. Before enabling force detach, non-graceful node shutdown (NGNS) must be enabled on the Kubernetes cluster. For more information, refer to Kubernetes: Non Graceful node shutdown.

When using the ontap-nas-economy driver, you need to set the autoExportPolicy parameter in the backend configuration to true so that Trident can restrict access from the Kubernetes node with the taint applied using managed export policies.

Because Trident relies on Kubernetes NGNS, do not remove out-of-service taints from an unhealthy node until all non-tolerable workloads are rescheduled. Recklessly applying or removing the taint can jeopardize backend data protection.

When the Kubernetes cluster administrator has applied the node.kubernetes.io/out-of-service=nodeshutdown:NoExecute taint to the node and enableForceDetach is set to true, Trident will determine the node status and:

Cease backend I/O access for volumes mounted to that node.

Mark the Trident node object as dirty (not safe for new publications).

The Trident controller will reject new publish volume requests until the node is re-qualified (after having been marked as dirty) by the Trident node pod. Any workloads scheduled with a mounted PVC (even after the cluster node is healthy and ready) will be not be accepted until Trident can verify the node clean (safe for new publications).

When node health is restored and the taint is removed, Trident will:

Identify and clean stale published paths on the node.
If the node is in a cleanable state (the out-of-service taint has been removed and the node is in Ready state) and all stale, published paths are clean, Trident will readmit the node as clean and allow new published volumes to the node.

Sample configurations

You can use the attributes in Configuration options when defining TridentOrchestrator to customize your installation.

Basic custom configuration

This is an example for a basic custom installation.

cat deploy/crds/tridentorchestrator_cr_imagepullsecrets.yaml
apiVersion: trident.netapp.io/v1
kind: TridentOrchestrator
metadata:
  name: trident
spec:
  debug: true
  namespace: trident
  imagePullSecrets:
  - thisisasecret

Node selectors

This example installs Trident with node selectors.

apiVersion: trident.netapp.io/v1
kind: TridentOrchestrator
metadata:
  name: trident
spec:
  debug: true
  namespace: trident
  controllerPluginNodeSelector:
    nodetype: master
  nodePluginNodeSelector:
    storage: netapp

Windows worker nodes

This example installs Trident on a Windows worker node.

cat deploy/crds/tridentorchestrator_cr.yaml
apiVersion: trident.netapp.io/v1
kind: TridentOrchestrator
metadata:
  name: trident
spec:
  debug: true
  namespace: trident
  windows: true

Managed identities on an AKS cluster

This example installs Trident to enable managed identities on an AKS cluster.

apiVersion: trident.netapp.io/v1
kind: TridentOrchestrator
metadata:
  name: trident
spec:
  debug: true
  namespace: trident
  cloudProvider: "Azure"

Cloud identity on an AKS cluster

This example installs Trident for use with a cloud identity on an AKS cluster.

apiVersion: trident.netapp.io/v1
kind: TridentOrchestrator
metadata:
  name: trident
spec:
  debug: true
  namespace: trident
  cloudProvider: "Azure"
  cloudIdentity: 'azure.workload.identity/client-id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx'

Cloud identity on an EKS cluster

This example installs Trident for use with a cloud identity on an AKS cluster.

apiVersion: trident.netapp.io/v1
kind: TridentOrchestrator
metadata:
  name: trident
spec:
  debug: true
  namespace: trident
  cloudProvider: "AWS"
  cloudIdentity: "'eks.amazonaws.com/role-arn: arn:aws:iam::123456:role/trident-role'"

Cloud identity for GKE

This example installs Trident for use with a cloud identity on a GKE cluster.

apiVersion: trident.netapp.io/v1
kind: TridentBackendConfig
metadata:
  name: backend-tbc-gcp-gcnv
spec:
  version: 1
  storageDriverName: google-cloud-netapp-volumes
  projectNumber: '012345678901'
  network: gcnv-network
  location: us-west2
  serviceLevel: Premium
  storagePool: pool-premium1

Customize Trident operator installation

Creating your file...

Understanding controller pods and node pods

Configuration options

Details about force detach

Sample configurations