Configure Google Cloud NetApp Volumes for SAN workloads
Use the google-cloud-netapp-volumes-san driver to provision iSCSI block volumes from Google Cloud NetApp Volumes Flex Unified storage pools.
|
|
The SAN driver supports block workloads only. To provision NAS volumes, use the google-cloud-netapp-volumes driver.
|
Overview
The driver provisions volumes from Flex Unified storage pools and presents them to Kubernetes nodes as iSCSI block devices.
The driver supports the following environments:
-
Trident 26.02 and later
-
Google Kubernetes Engine (GKE), or Red Hat OpenShift clusters running on GCP
-
Google Cloud NetApp Volumes Flex Unified storage pools
-
iSCSI-based workloads
Starting with Trident 26.10, the SAN driver supports Google Cloud Workload Identity Federation (WIF) on self-managed OpenShift Container Platform (OCP, including IPI deployments) and OpenShift Dedicated (OSD) clusters running on GCP. For WIF prerequisites and backend configuration, see Use Workload Identity Federation on OpenShift. Use the SAN driver name and Flex Unified storage settings on this page.
Configure WIF for SAN
WIF exchanges a projected Kubernetes service account token for short-lived Google Cloud credentials. Trident uses these credentials to impersonate a Google Cloud service account when it calls the Google Cloud NetApp Volumes API.
Configure a Google Cloud Workload Identity Pool and OIDC provider for the OpenShift cluster. The provider issuer must match the token's iss claim, and its attribute mapping must map assertion.sub to google.subject.
Install Trident with the provider resource name as the WIF audience. This causes Trident to project the controller service account token at /var/run/secrets/wif-token/token.
apiVersion: trident.netapp.io/v1
kind: TridentOrchestrator
metadata:
name: trident
spec:
namespace: trident
cloudProvider: GCP
gcpWIFAudience: "//iam.googleapis.com/projects/<wif-pool-project-number>/locations/global/workloadIdentityPools/<pool-id>/providers/<provider-id>"
helm install trident <trident-operator-package> \
--namespace trident \
--create-namespace \
--set cloudProvider=GCP \
--set-string gcpWIFAudience="//iam.googleapis.com/projects/<wif-pool-project-number>/locations/global/workloadIdentityPools/<pool-id>/providers/<provider-id>"
tridentctl install \
--namespace trident \
--cloud-provider GCP \
--gcp-wif-audience "//iam.googleapis.com/projects/<wif-pool-project-number>/locations/global/workloadIdentityPools/<pool-id>/providers/<provider-id>"
Use the same audience for the Trident installation and wipCredentialConfig.audience. Do not also configure GKE cloudIdentity or --cloud-identity. Do not patch the controller Deployment to add the projected token; the Trident operator can overwrite that change.
|
|
On OpenShift Dedicated, the OIDC provider can be shared with other OpenShift components. If the provider's |
Grant the WIF principal for the Trident controller service account the roles/iam.workloadIdentityUser role on the Google Cloud service account that Trident will impersonate. For the default Trident controller service account, the token subject is system:serviceaccount:trident:trident-controller; use the subject configured for your installation. Grant the impersonated service account roles/netapp.admin, or equivalent permissions, on the project containing the Google Cloud NetApp Volumes resources.
Add the following stanza to the spec of the SAN backend:
wipCredentialConfig:
type: external_account
audience: "//iam.googleapis.com/projects/<wif-pool-project-number>/locations/global/workloadIdentityPools/<pool-id>/providers/<provider-id>"
subject_token_type: urn:ietf:params:oauth:token-type:jwt
token_url: https://sts.googleapis.com/v1/token
credential_source:
file: /var/run/secrets/wif-token/token
service_account_impersonation_url: https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/<service-account-email>:generateAccessToken
The WIF configuration is the same for both GCNV drivers. The SAN backend must still use google-cloud-netapp-volumes-san and a supported Flex Unified storage pool.
Flex Unified storage pools
Flex Unified storage pools provide block storage using the iSCSI protocol and are required for SAN provisioning:
-
Flex Unified REGIONAL pools are supported.
-
Flex Unified ZONAL pools are supported starting with Trident 26.02.1.
-
Only the Flex service level is supported for SAN workloads.
The backend location must match the region of the Flex Unified pool. If Trident does not find the pool, verify the region, network, and pool resource path.
Configure a Trident SAN backend
apiVersion: trident.netapp.io/v1
kind: TridentBackendConfig
metadata:
name: gcnv-san
namespace: trident
spec:
version: 1
storageDriverName: google-cloud-netapp-volumes-san
projectNumber: "<project-number>"
location: "<region>"
sdkTimeout: "600"
storage:
- labels:
cloud: gcp
performance: flex
network: "<vpc-network>"
serviceLevel: Flex
Create a StorageClass
After configuring the SAN backend, create a StorageClass that references the google-cloud-netapp-volumes-san driver.
Specify the filesystem type in the StorageClass, not in the backend.
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: gcnv-san
provisioner: csi.trident.netapp.io
parameters:
backendType: "google-cloud-netapp-volumes-san"
fsType: "ext4"
allowVolumeExpansion: true
The driver supports the following filesystem types:
-
ext4(default) -
ext3 -
xfs
|
|
The SAN driver supports only the Flex service level and does not use NAS-specific backend parameters such as exportRule, unixPermissions, nasType, snapshotDir, nfsMountOptions, or tiering-related settings.
|
Verify WIF provisioning
Apply the backend and verify that the TridentBackendConfig is bound and its last operation succeeded:
oc apply -f <backend-file>.yaml
oc -n trident get tridentbackendconfig <backend-name> -o yaml
Create a PVC using the gcnv-san StorageClass on this page and verify that it reaches Bound. This validates the WIF exchange, service account impersonation, Google Cloud NetApp Volumes permissions, and SAN pool selection together. Delete the test PVC when you finish.
If backend reconciliation fails, inspect the Trident controller logs:
oc -n trident logs deploy/trident-controller -c trident-main --tail=400
Check the following for common errors:
-
For an
invalid_grantaudience error, compare the token'saudclaim with the OIDC provider's allowed audiences andwipCredentialConfig.audience. -
For an
iam.serviceAccounts.getAccessTokenpermission error, verify that the Trident controller's WIF principal hasroles/iam.workloadIdentityUseron the impersonated service account. -
For a
netapp.storagePools.listpermission error, verify that the impersonated service account has the required Google Cloud NetApp Volumes permissions on the correct project. -
If the pool is not found, verify that its location and type are supported, the backend region matches the pool region, and the backend references the correct network and pool.
Provision block volumes
ReadWriteOnce (RWO)
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: gcnv-san-rwo
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 100Gi
storageClassName: gcnv-san
ReadWriteOncePod (RWOP)
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: gcnv-san-rwop
spec:
accessModes:
- ReadWriteOncePod
resources:
requests:
storage: 100Gi
storageClassName: gcnv-san
ReadOnlyMany (ROX)
A common pattern for ROX is to clone an existing ReadWriteOnce volume and mount the clone as read-only.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: gcnv-san-rox
spec:
accessModes:
- ReadOnlyMany
resources:
requests:
storage: 100Gi
storageClassName: gcnv-san
dataSource:
kind: PersistentVolumeClaim
name: gcnv-san-rwo
ReadWriteMany (RWX) — raw block only
ReadWriteMany is supported only when volumeMode: Block.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: gcnv-san-raw-rwx
spec:
accessModes:
- ReadWriteMany
volumeMode: Block
resources:
requests:
storage: 100Gi
storageClassName: gcnv-san
Block volume behavior
Block volumes are provisioned as iSCSI LUNs and presented to Kubernetes nodes as block devices.
Block volumes:
-
Use the iSCSI protocol
-
Support filesystem and raw block presentation
-
Are attached and managed by Trident
-
Support multiple Kubernetes access modes
Access modes
Block volumes provisioned by Trident support the following access modes:
-
ReadWriteOnce(RWO) -
ReadOnlyMany(ROX) -
ReadWriteOncePod(RWOP) -
ReadWriteMany(RWX), supported only whenvolumeMode: Block
volumeMode behavior
The volumeMode field controls how a block volume is exposed:
-
Filesystem
Trident formats and mounts the volume. -
Block
Trident attaches the device and exposes it as a raw block device.
Supported operations
Block volumes provisioned using the google-cloud-netapp-volumes-san driver support:
-
Create
-
Delete
-
Clone
-
Snapshot
-
Resize
-
Import
Volume size overhead
Google Cloud NetApp Volumes block volumes include internal metadata overhead. As a result, the device size visible to the kernel is smaller than the provisioned capacity.
Observed overhead:
-
Approximately 300 KiB when a volume is created
-
Up to approximately 107 MiB after a resize
Google Cloud NetApp Volumes accepts capacity allocations in whole GiB. To ensure that the usable device size meets or exceeds the PVC request, Trident:
-
Rounds the requested size up to the next whole GiB.
-
Adds a 1 GiB buffer.
For example:
-
PVC request: 100 GiB.
-
Provisioned capacity: 101 GiB.
-
Usable capacity visible to the application: at least 100 GiB.
Pod examples
Filesystem-mounted block volume (RWO)
apiVersion: v1
kind: Pod
metadata:
name: app-rwo
spec:
containers:
- name: app
image: ubuntu:22.04
command: ["sleep", "infinity"]
volumeMounts:
- name: data
mountPath: /mnt/data
volumes:
- name: data
persistentVolumeClaim:
claimName: gcnv-san-rwo
Raw block device (RWX)
apiVersion: v1
kind: Pod
metadata:
name: app-raw-rwx
spec:
containers:
- name: app
image: ubuntu:22.04
command: ["sleep", "infinity"]
volumeDevices:
- name: data
devicePath: /dev/xda
volumes:
- name: data
persistentVolumeClaim:
claimName: gcnv-san-raw-rwx
Attach and mount behavior
For SAN volumes provisioned from Google Cloud NetApp Volumes:
-
Trident creates a Logical Unit Number (LUN) in a Flex Unified storage pool.
-
During publish, Trident maps the LUN to a per-node host group.
-
During node staging, Trident:
-
Logs in to the iSCSI target
-
Discovers the LUN
-
Configures multipath
-
-
If
volumeMode: Filesystem, Trident formats the device if required and mounts it. -
If
volumeMode: Block, Trident attaches the device and exposes it directly to the pod without formatting or mounting.
|
|
SAN block volumes do not provide distributed locking or write coordination. When a block volume is accessed by multiple nodes (ReadWriteMany with volumeMode: Block), the application or filesystem must manage concurrency.
|