Skip to main content
Cloud Manager 3.8
A newer release of this product is available.

Using Cloud Volumes ONTAP as persistent storage for Kubernetes

Contributors netapp-bcammett

Cloud Manager can automate the deployment of NetApp Trident on Kubernetes clusters so you can use Cloud Volumes ONTAP as persistent storage for containers.

Trident is a fully-supported open source project maintained by NetApp. Trident integrates natively with Kubernetes and its Persistent Volume framework to seamlessly provision and manage volumes from systems running any combination of NetApp's storage platforms. Learn more about Trident.

Note The Kubernetes feature isn't supported with on-prem ONTAP clusters. It's supported with Cloud Volumes ONTAP only.

Quick start

Get started quickly by following these steps or scroll down to the remaining sections for full details.

Number 1 Review prerequisites

Ensure that your environment can meet the prerequisites, which includes connectivity between Kubernetes clusters and Cloud Volumes ONTAP, connectivity between Kubernetes clusters and a Connector, a minimum Kubernetes version of 1.14, at least one worker node in a cluster, and more. See the complete list.

Number 2 Add your Kubernetes clusters to Cloud Manager

In Cloud Manager, click Kubernetes and discover clusters directly from your cloud provider's managed service or import a cluster by providing a kubeconfig file.

Number 3 Connect your clusters to Cloud Volumes ONTAP

After you add a Kubernetes cluster, click Connect to Working Environment to connect the cluster to one or more Cloud Volumes ONTAP systems.

Number 4 Start provisioning Persistent Volumes

Request and manage Persistent Volumes using native Kubernetes interfaces and constructs. Cloud Manager creates NFS and iSCSI storage classes that you can use when provisioning Persistent Volumes.

Learn more about provisioning your first volume with Trident for Kubernetes.

Reviewing prerequisites

Before you get started, ensure that your Kubernetes clusters and Connector meet specific requirements.

Kubernetes cluster requirements

  • Network connectivity is required between a Kubernetes cluster and the Connector and between a Kubernetes cluster and Cloud Volumes ONTAP.

    Both the Connector and Cloud Volumes ONTAP need a connection to the Kubernetes API endpoint:

    • For managed clusters, set up a route between a cluster's VPC and the VPC where the Connector and Cloud Volumes ONTAP reside.

    • For other clusters, the IP address of the master node or load balancer (as listed in the kubeconfig file) must be reachable by the Connector and Cloud Volumes ONTAP, and it must present a valid TLS certificate.

  • A Kubernetes cluster can be in any location that has the network connectivity listed above.

  • A Kubernetes cluster must be running version 1.14 at a minimum.

    The maximum supported version is defined by Trident. Click here to see the maximum supported Kubernetes version.

  • A Kubernetes cluster must have at least one worker node.

  • For clusters running in Amazon Elastic Kubernetes Service (Amazon EKS), each cluster needs an IAM role added in order to resolve a permissions error. After you add the cluster, Cloud Manager will prompt you with the exact eksctl command that resolves the error.

    Learn about IAM permissions boundaries.

  • For clusters running in Azure Kubernetes Service (AKS), those clusters must be assigned the Azure Kubernetes Service RBAC Cluster Admin role. This is required so Cloud Manager can install Trident and configure storage classes on the cluster.

  • For clusters running in Google Kubernetes Engine (GKE), those clusters must not use the default Container Optimized OS. You should switch them to use Ubuntu.

    GKE defaults to using the Google container-optimized image, which doesn't have the utilities that Trident needs to mount volumes.

Connector requirements

Ensure that the following networking and permissions are in place for the Connector.

Networking

  • The Connector needs an outbound internet connection to access the following endpoints when installing Trident:

    https://packages.cloud.google.com/yum
    https://github.com/NetApp/trident/releases/download/

    Cloud Manager installs Trident on a Kubernetes cluster when you connect a working environment to the cluster.

Required permissions to discover and manage EKS clusters

The Connector needs Admin permissions to discover and manage Kubernetes clusters running in Amazon Elastic Kubernetes Service (EKS):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "eks:*",
            "Resource": "*"
       }
    ]
}

Required permissions to discover and manage GKE clusters

The Connector needs the following permissions to discover and manage Kubernetes clusters running in Google Kubernetes Engine (GKE):

container.*

Example setup

The following image shows an example of a Kubernetes cluster running in Amazon Elastic Kubernetes Service (Amazon EKS) and its connections to the Connector and Cloud Volumes ONTAP.

A architectural diagram of a Kubernetes cluster running in AWS and its connection to a Connecter and Cloud Volumes ONTAP which are also running in AWS.

Adding Kubernetes clusters

Add Kubernetes clusters to Cloud Manager by discovering the clusters running in your cloud provider's managed Kubernetes service or by importing a cluster's kubeconfig file.

Steps

  1. At the top of Cloud Manager, click Kubernetes.

  2. Click Add Cluster.

  3. Choose one of the available options:

    • Click Discover Clusters to discover the managed clusters that Cloud Manager has access to based on permissions that you provided to the Connector.

      For example, if your Connector is running in Google Cloud, Cloud Manager uses the permissions from the Connector's service account to discover clusters running in Google Kubernetes Engine (GKE).

    • Click Import Cluster to import a cluster using a kubeconfig file.

      After you upload the file, Cloud Manager verifies connectivity to the cluster and saves an encrypted copy of the kubeconfig file.

Result

Cloud Manager adds the Kubernetes cluster. You can now connect the cluster to Cloud Volumes ONTAP.

Connecting a cluster to Cloud Volumes ONTAP

Connect a Kubernetes cluster to Cloud Volumes ONTAP so you can use Cloud Volumes ONTAP as persistent storage for containers.

Steps

  1. At the top of Cloud Manager, click Kubernetes.

  2. Click Connect to Working Environment for the cluster that you just added.

    A screenshot of the Kubernetes cluster list where you can click Connect to Working Environment.

  3. Select a working environment and click Continue.

  4. Choose the NetApp storage class to use as the default storage class for the Kubernetes cluster and click Continue.

    When a user creates a persistent volume, the Kubernetes cluster can use this storage class as the backend storage by default.

  5. Choose whether to use default auto export policies or whether to add a custom CIDR block.

    A screenshot of the Confirm page where you review your options and set up an export policy.

  6. Click Add Working Environment.

Result

Cloud Manager connects the working environment to the cluster, which can take up to 15 minutes.

Managing your clusters

Cloud Manager enables you to manage your Kubernetes clusters by changing the default storage class, upgrading Trident, and more.

Changing the default storage class

Make sure that you've set a Cloud Volumes ONTAP storage class as the default storage class so clusters use Cloud Volumes ONTAP as the backend storage.

Steps

  1. At the top of Cloud Manager, click Kubernetes.

  2. Click the name of the Kubernetes cluster.

  3. In the Storage Classes table, click the actions menu on the far right for the storage class that you'd like to set as the default.

    A screenshot of the Storage Classes table where you can click the action menu and select Set as Default.

  4. Click Set as Default.

Upgrading Trident

You can upgrade Trident from Cloud Manager when a new version of Trident is available.

Steps

  1. At the top of Cloud Manager, click Kubernetes.

  2. Click the name of the Kubernetes cluster.

  3. If a new version is available, click Upgrade next to the Trident version.

    A screenshot of the Cluster Details page where the Upgrade button appears next to the Trident version.

Updating the kubeconfig file

If you added your cluster to Cloud Manager by importing the kubeconfig file, you can upload the latest kubeconfig file to Cloud Manager at any time. You might do this if you've updated the credentials, if you've changed users or roles, or if something changed that affects the cluster, user, namespaces, or authentication.

Steps

  1. At the top of Cloud Manager, click Kubernetes.

  2. Click the name of the Kubernetes cluster.

  3. Click Update Kubeconfig.

  4. When prompted through your web browser, select the updated kubeconfig file and click Open.

Result

Cloud Manager updates information about the Kubernetes cluster based on the latest kubeconfig file.

Disconnecting a cluster

When you disconnect a cluster from Cloud Volumes ONTAP, you can no longer use that Cloud Volumes ONTAP system as persistent storage for containers. Existing Persistent Volumes are not deleted.

Steps

  1. At the top of Cloud Manager, click Kubernetes.

  2. Click the name of the Kubernetes cluster.

  3. In the Working Environments table, click the actions menu on the far right for the working environment that you want to disconnect.

    A screenshot of the Working Environments table where the Disconnect action appears after you click the menu in the far right of the table.

  4. Click Disconnect.

Result

Cloud Manager disconnects the cluster from the Cloud Volumes ONTAP system.

Removing a cluster

Remove decommissioned clusters from Cloud Manager after you disconnect all working environments from the cluster.

Steps

  1. At the top of Cloud Manager, click Kubernetes.

  2. Click the name of the Kubernetes cluster.

  3. Click Remove Cluster.

    A screenshot of the Remove Cluster button that appears at the top of the cluster details page.