Skip to main content

Configure Google Cloud NetApp Volumes for SAN workloads

Contributors joan-ing

Use the google-cloud-netapp-volumes-san driver to provision iSCSI block volumes from Google Cloud NetApp Volumes Flex Unified storage pools.

Note The SAN driver supports block workloads only. To provision NAS volumes, use the google-cloud-netapp-volumes driver.

Overview

The driver provisions volumes from Flex Unified storage pools and presents them to Kubernetes nodes as iSCSI block devices.

The driver supports the following environments:

  • Trident 26.02 and later

  • Google Kubernetes Engine (GKE), or Red Hat OpenShift clusters running on GCP

  • Google Cloud NetApp Volumes Flex Unified storage pools

  • iSCSI-based workloads

Starting with Trident 26.10, the SAN driver supports Google Cloud Workload Identity Federation (WIF) on self-managed OpenShift Container Platform (OCP, including IPI deployments) and OpenShift Dedicated (OSD) clusters running on GCP. For WIF prerequisites and backend configuration, see Use Workload Identity Federation on OpenShift. Use the SAN driver name and Flex Unified storage settings on this page.

Configure WIF for SAN

WIF exchanges a projected Kubernetes service account token for short-lived Google Cloud credentials. Trident uses these credentials to impersonate a Google Cloud service account when it calls the Google Cloud NetApp Volumes API.

Configure a Google Cloud Workload Identity Pool and OIDC provider for the OpenShift cluster. The provider issuer must match the token's iss claim, and its attribute mapping must map assertion.sub to google.subject.

Install Trident with the provider resource name as the WIF audience. This causes Trident to project the controller service account token at /var/run/secrets/wif-token/token.

Trident operator
apiVersion: trident.netapp.io/v1
kind: TridentOrchestrator
metadata:
  name: trident
spec:
  namespace: trident
  cloudProvider: GCP
  gcpWIFAudience: "//iam.googleapis.com/projects/<wif-pool-project-number>/locations/global/workloadIdentityPools/<pool-id>/providers/<provider-id>"
Helm
helm install trident <trident-operator-package> \
  --namespace trident \
  --create-namespace \
  --set cloudProvider=GCP \
  --set-string gcpWIFAudience="//iam.googleapis.com/projects/<wif-pool-project-number>/locations/global/workloadIdentityPools/<pool-id>/providers/<provider-id>"
tridentctl
tridentctl install \
  --namespace trident \
  --cloud-provider GCP \
  --gcp-wif-audience "//iam.googleapis.com/projects/<wif-pool-project-number>/locations/global/workloadIdentityPools/<pool-id>/providers/<provider-id>"

Use the same audience for the Trident installation and wipCredentialConfig.audience. Do not also configure GKE cloudIdentity or --cloud-identity. Do not patch the controller Deployment to add the projected token; the Trident operator can overwrite that change.

Important

On OpenShift Dedicated, the OIDC provider can be shared with other OpenShift components. If the provider's allowedAudiences list is empty, Google accepts the provider's canonical resource name, with or without the https:// prefix, so no provider change is needed for the audience above. Do not set an explicit audience list only for Trident. If the provider already has explicit audiences, append the Trident audience while preserving every existing value. Replacing the list can prevent other OpenShift components from exchanging tokens.

Grant the WIF principal for the Trident controller service account the roles/iam.workloadIdentityUser role on the Google Cloud service account that Trident will impersonate. For the default Trident controller service account, the token subject is system:serviceaccount:trident:trident-controller; use the subject configured for your installation. Grant the impersonated service account roles/netapp.admin, or equivalent permissions, on the project containing the Google Cloud NetApp Volumes resources.

Add the following stanza to the spec of the SAN backend:

wipCredentialConfig:
  type: external_account
  audience: "//iam.googleapis.com/projects/<wif-pool-project-number>/locations/global/workloadIdentityPools/<pool-id>/providers/<provider-id>"
  subject_token_type: urn:ietf:params:oauth:token-type:jwt
  token_url: https://sts.googleapis.com/v1/token
  credential_source:
    file: /var/run/secrets/wif-token/token
  service_account_impersonation_url: https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/<service-account-email>:generateAccessToken

The WIF configuration is the same for both GCNV drivers. The SAN backend must still use google-cloud-netapp-volumes-san and a supported Flex Unified storage pool.

Flex Unified storage pools

Flex Unified storage pools provide block storage using the iSCSI protocol and are required for SAN provisioning:

  • Flex Unified REGIONAL pools are supported.

  • Flex Unified ZONAL pools are supported starting with Trident 26.02.1.

  • Only the Flex service level is supported for SAN workloads.

The backend location must match the region of the Flex Unified pool. If Trident does not find the pool, verify the region, network, and pool resource path.

Configure a Trident SAN backend

apiVersion: trident.netapp.io/v1
kind: TridentBackendConfig
metadata:
  name: gcnv-san
  namespace: trident
spec:
  version: 1
  storageDriverName: google-cloud-netapp-volumes-san
  projectNumber: "<project-number>"
  location: "<region>"
  sdkTimeout: "600"
  storage:
  - labels:
      cloud: gcp
      performance: flex
    network: "<vpc-network>"
    serviceLevel: Flex

Create a StorageClass

After configuring the SAN backend, create a StorageClass that references the google-cloud-netapp-volumes-san driver.

Specify the filesystem type in the StorageClass, not in the backend.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: gcnv-san
provisioner: csi.trident.netapp.io
parameters:
  backendType: "google-cloud-netapp-volumes-san"
  fsType: "ext4"
allowVolumeExpansion: true

The driver supports the following filesystem types:

  • ext4 (default)

  • ext3

  • xfs

Note The SAN driver supports only the Flex service level and does not use NAS-specific backend parameters such as exportRule, unixPermissions, nasType, snapshotDir, nfsMountOptions, or tiering-related settings.

Verify WIF provisioning

Apply the backend and verify that the TridentBackendConfig is bound and its last operation succeeded:

oc apply -f <backend-file>.yaml
oc -n trident get tridentbackendconfig <backend-name> -o yaml

Create a PVC using the gcnv-san StorageClass on this page and verify that it reaches Bound. This validates the WIF exchange, service account impersonation, Google Cloud NetApp Volumes permissions, and SAN pool selection together. Delete the test PVC when you finish.

If backend reconciliation fails, inspect the Trident controller logs:

oc -n trident logs deploy/trident-controller -c trident-main --tail=400

Check the following for common errors:

  • For an invalid_grant audience error, compare the token's aud claim with the OIDC provider's allowed audiences and wipCredentialConfig.audience.

  • For an iam.serviceAccounts.getAccessToken permission error, verify that the Trident controller's WIF principal has roles/iam.workloadIdentityUser on the impersonated service account.

  • For a netapp.storagePools.list permission error, verify that the impersonated service account has the required Google Cloud NetApp Volumes permissions on the correct project.

  • If the pool is not found, verify that its location and type are supported, the backend region matches the pool region, and the backend references the correct network and pool.

Provision block volumes

ReadWriteOnce (RWO)

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: gcnv-san-rwo
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 100Gi
  storageClassName: gcnv-san

ReadWriteOncePod (RWOP)

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: gcnv-san-rwop
spec:
  accessModes:
    - ReadWriteOncePod
  resources:
    requests:
      storage: 100Gi
  storageClassName: gcnv-san

ReadOnlyMany (ROX)

A common pattern for ROX is to clone an existing ReadWriteOnce volume and mount the clone as read-only.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: gcnv-san-rox
spec:
  accessModes:
    - ReadOnlyMany
  resources:
    requests:
      storage: 100Gi
  storageClassName: gcnv-san
  dataSource:
    kind: PersistentVolumeClaim
    name: gcnv-san-rwo

ReadWriteMany (RWX) — raw block only

ReadWriteMany is supported only when volumeMode: Block.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: gcnv-san-raw-rwx
spec:
  accessModes:
    - ReadWriteMany
  volumeMode: Block
  resources:
    requests:
      storage: 100Gi
  storageClassName: gcnv-san

Block volume behavior

Block volumes are provisioned as iSCSI LUNs and presented to Kubernetes nodes as block devices.

Block volumes:

  • Use the iSCSI protocol

  • Support filesystem and raw block presentation

  • Are attached and managed by Trident

  • Support multiple Kubernetes access modes

Access modes

Block volumes provisioned by Trident support the following access modes:

  • ReadWriteOnce (RWO)

  • ReadOnlyMany (ROX)

  • ReadWriteOncePod (RWOP)

  • ReadWriteMany (RWX), supported only when volumeMode: Block

volumeMode behavior

The volumeMode field controls how a block volume is exposed:

  • Filesystem
    Trident formats and mounts the volume.

  • Block
    Trident attaches the device and exposes it as a raw block device.

Supported operations

Block volumes provisioned using the google-cloud-netapp-volumes-san driver support:

  • Create

  • Delete

  • Clone

  • Snapshot

  • Resize

  • Import

Volume size overhead

Google Cloud NetApp Volumes block volumes include internal metadata overhead. As a result, the device size visible to the kernel is smaller than the provisioned capacity.

Observed overhead:

  • Approximately 300 KiB when a volume is created

  • Up to approximately 107 MiB after a resize

Google Cloud NetApp Volumes accepts capacity allocations in whole GiB. To ensure that the usable device size meets or exceeds the PVC request, Trident:

  • Rounds the requested size up to the next whole GiB.

  • Adds a 1 GiB buffer.

For example:

  • PVC request: 100 GiB.

  • Provisioned capacity: 101 GiB.

  • Usable capacity visible to the application: at least 100 GiB.

Pod examples

Filesystem-mounted block volume (RWO)

apiVersion: v1
kind: Pod
metadata:
  name: app-rwo
spec:
  containers:
  - name: app
    image: ubuntu:22.04
    command: ["sleep", "infinity"]
    volumeMounts:
    - name: data
      mountPath: /mnt/data
  volumes:
  - name: data
    persistentVolumeClaim:
      claimName: gcnv-san-rwo

Raw block device (RWX)

apiVersion: v1
kind: Pod
metadata:
  name: app-raw-rwx
spec:
  containers:
  - name: app
    image: ubuntu:22.04
    command: ["sleep", "infinity"]
    volumeDevices:
    - name: data
      devicePath: /dev/xda
  volumes:
  - name: data
    persistentVolumeClaim:
      claimName: gcnv-san-raw-rwx

Attach and mount behavior

For SAN volumes provisioned from Google Cloud NetApp Volumes:

  • Trident creates a Logical Unit Number (LUN) in a Flex Unified storage pool.

  • During publish, Trident maps the LUN to a per-node host group.

  • During node staging, Trident:

    • Logs in to the iSCSI target

    • Discovers the LUN

    • Configures multipath

  • If volumeMode: Filesystem, Trident formats the device if required and mounts it.

  • If volumeMode: Block, Trident attaches the device and exposes it directly to the pod without formatting or mounting.

Note SAN block volumes do not provide distributed locking or write coordination. When a block volume is accessed by multiple nodes (ReadWriteMany with volumeMode: Block), the application or filesystem must manage concurrency.