Skip to main content

Controller scalability

Contributors joan-ing
PDFs

Trident introduces controller scalability through improved concurrency across multiple storage drivers.
Customers can identify which Trident drivers support controller scalability at general availability and which drivers are available as a technical preview in Trident 26.02.
This enables informed deployment decisions and appropriate risk management for scalable Kubernetes environments.

Key concepts and definitions

Controller scalability

Controller scalability refers to the Trident controller's ability to process multiple storage operations in parallel rather than serializing them behind a single lock.
These operations include volume creation, deletion, resizing, snapshot creation and deletion, volume publish and unpublish, and backend management.

When controller scalability is enabled, operations on different volumes and backends proceed concurrently.
This increases throughput and reduces end-to-end operation time in environments with high numbers of concurrent PersistentVolumeClaim and VolumeSnapshot operations.

Controller scalability support

Trident supports controller scalability with different maturity levels depending on the storage driver.

General availability

The following drivers support controller scalability at general availability in Trident 26.02:

  • san

  • nas

  • san-nvme

  • google-cloud-netapp-volumes

Enable controller scalability

Controller scalability is controlled by the enableConcurrency configuration option. This option must be explicitly enabled during Trident installation or by updating an existing deployment.

Trident operator deployment

To enable controller scalability with the Trident operator, set enableConcurrency to true in the TridentOrchestrator custom resource.

New installation

Create or edit the TridentOrchestrator CR with enableConcurrency set to true:

apiVersion: trident.netapp.io/v1
kind: TridentOrchestrator
metadata:
  name: trident
spec:
  namespace: trident
  enableConcurrency: true

Apply the CR:

kubectl apply -f tridentorchestrator_cr.yaml

Existing installation

Patch the existing TridentOrchestrator CR to enable controller scalability:

kubectl patch torc trident --type=merge -p '{"spec":{"enableConcurrency":true}}'

Verify the setting was applied:

kubectl get torc trident -o jsonpath='{.status.currentInstallationParams.enableConcurrency}'

Helm deployment

To enable controller scalability with Helm, set the enableConcurrency value to true.

New installation

helm install trident netapp-trident/trident-operator --namespace trident --create-namespace --set enableConcurrency=true

Existing installation

helm upgrade trident netapp-trident/trident-operator --namespace trident --set enableConcurrency=true

Alternatively, set enableConcurrency to true in a custom values.yaml file:

# values.yaml
enableConcurrency: true

Then install or upgrade using the values file:

helm install trident netapp-trident/trident-operator --namespace trident --create-namespace -f values.yaml

tridentctl deployment

To enable controller scalability with tridentctl, pass the --enable-concurrency flag during installation.

New installation

tridentctl install -n trident --enable-concurrency

Existing installation

To enable controller scalability on an existing tridentctl-based deployment, uninstall and reinstall with the flag:

tridentctl uninstall -n trident
tridentctl install -n trident --enable-concurrency

Verify controller scalability is enabled

After enabling controller scalability, verify that the Trident controller is running with concurrency enabled by checking the controller pod logs:

kubectl logs -n trident deploy/trident-controller | grep -i concurrency

You should see a log entry indicating that concurrency is enabled.

Technical preview

The following drivers support controller scalability as a technical preview in Trident 26.02:

  • nas-eco

  • san-eco

For these drivers:

  • Controller concurrency is available for evaluation and testing

  • Behavior may change in future releases

  • Use in production environments is not recommended

Concurrency behavior

When controller scalability is enabled:

  • Trident replaces the single global lock with fine-grained, per-resource locking

  • Operations that modify the same resource are serialized to maintain data consistency

  • Operations that only read from a resource can proceed concurrently with other read operations on that resource

  • Trident limits concurrent ONTAP API requests to 20 per management LIF to prevent overloading backend storage systems

  • If multiple backends share the same management LIF, they share this 20-request limit

Known limitations and considerations

The following considerations apply to controller scalability:

  • Concurrency is managed internally by the Trident controller

  • There are no user-configurable concurrency limits in this release

  • Overall throughput depends on:

    • The storage driver in use

    • Backend responsiveness

    • Kubernetes API server performance

  • High concurrency can increase load on backend storage systems

Caveats and limitations

The following limitations apply in Trident 26.02:

  • Controller scalability behavior is not identical across all drivers

  • Technical preview drivers may exhibit:

    • Inconsistent performance under high load

    • Changes in behavior between releases

  • Debugging concurrent operations may be more complex due to parallel execution

  • Metrics and logs may show interleaved operation output

Recommendations

  • Use general availability (GA) drivers for production environments that require high scalability

  • Evaluate technical preview drivers in non-production environments

  • Monitor backend and controller performance when operating at scale

  • Avoid assuming operation ordering in automation scripts