Requirements for Kubernetes clusters in AWS

You can add managed Amazon Elastic Kubernetes Service (EKS) clusters or self-managed Kubernetes clusters on AWS to BlueXP. Before you can add the clusters to BlueXP, you need to ensure that the following requirements are met.

Note This topic uses Kubernetes cluster where configuration is the same for EKS and self-managed Kubernetes clusters. The cluster type is specified where configuration differs.

Requirements

Astra Trident

One of the four most recent versions of Astra Trident is required. You can install or upgrade Astra Trident directly from BlueXP. You should review the prerequisites prior to installing Astra Trident.

Cloud Volumes ONTAP

Cloud Volumes ONTAP for AWS must be set up as backend storage for the cluster. Go to the Astra Trident docs for configuration steps.

BlueXP Connector

A Connector must be running in AWS with the required permissions. Learn more below.

Network connectivity

Network connectivity is required between the Kubernetes cluster and the Connector and between the Kubernetes cluster and Cloud Volumes ONTAP. Learn more below.

RBAC authorization

The BlueXP Connector role must be authorized on each Kubernetes cluster. Learn more below.

Prepare a Connector

A BlueXP Connector is required in AWS to discover and manage Kubernetes clusters. You’ll need to create a new Connector or use an existing Connector that has the required permissions.

Create a new Connector

Follow the steps in one of the links below.

Add the required permissions to an existing Connector

Starting in the 3.9.13 release, any newly created Connectors include three new AWS permissions that enable discovery and management of Kubernetes clusters. If you created a Connector prior to this release, then you’ll need to modify the existing policy for the Connector’s IAM role to provide the permissions.

Steps

  1. Go the AWS console and open the EC2 service.

  2. Select the Connector instance, click Security, and click the name of the IAM role to view the role in the IAM service.

    A screenshot of the AWS console that shows the name of the IAM role in the Security tab.

  3. In the Permissions tab, expand the policy and click Edit policy.

    A screenshot of the AWS console that shows the Edit policy button in the Permissions tab.

  4. Click JSON and add the following permissions under the first set of actions:

    • ec2:DescribeRegions

    • eks:ListClusters

    • eks:DescribeCluster

    • iam:GetInstanceProfile

    View the full JSON format for the policy

  5. Click Review policy and then click Save changes.

Review networking requirements

You need to provide network connectivity between the Kubernetes cluster and the Connector and between the Kubernetes cluster and the Cloud Volumes ONTAP system that provides backend storage to the cluster.

  • Each Kubernetes cluster must have an inbound connection from the Connector

  • The Connector must have an outbound connection to each Kubernetes cluster over port 443

The simplest way to provide this connectivity is to deploy the Connector and Cloud Volumes ONTAP in the same VPC as the Kubernetes cluster. Otherwise, you need to set up a VPC peering connection between the different VPCs.

Here’s an example that shows each component in the same VPC.

An architectural diagram of an EKS Kubernetes cluster and its connection to a Connecter and Cloud Volumes ONTAP in the same VPC.

And here’s another example that shows an EKS cluster running in a different VPC. In this example, VPC peering provides a connection between the VPC for the EKS cluster and the VPC for the Connector and Cloud Volumes ONTAP.

An architectural diagram of an EKS Kubernetes cluster and its connection to a Connecter and Cloud Volumes ONTAP in a separate VPC.

Set up RBAC authorization

You need to authorize the Connector role on each Kubernetes cluster so the Connector can discover and manage a cluster.

Different authorization is required to enable different functionality.

Backup and restore

Backup and restore requires only basic authorization.

Add storage classes

Expanded authorization is required to add storage classes using BlueXP and monitor the cluster for changes to the backend.

Install Astra trident

You need to provide full authorization for BlueXP to install Astra Trident.

Note When installing Astra Trident, BlueXP installs the Astra Trident backend and Kubernetes secret that contains the credentials Astra Trident needs to communicate with the storage cluster.
Steps

  1. Create a cluster role and role binding.

    1. You can customize authorization based on your requirements.

      Backup/restore

      Add basic authorization to enable backup and restore for Kubernetes clusters.

      apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
    name: cloudmanager-access-clusterrole
rules:
    - apiGroups:
          - ''
      resources:
          - namespaces
      verbs:
          - list
          - watch
    - apiGroups:
          - ''
      resources:
          - persistentvolumes
      verbs:
          - list
          - watch
    - apiGroups:
          - ''
      resources:
          - pods
          - pods/exec
      verbs:
          - get
          - list
          - watch
    - apiGroups:
          - ''
      resources:
          - persistentvolumeclaims
      verbs:
          - list
          - create
          - watch
    - apiGroups:
          - storage.k8s.io
      resources:
          - storageclasses
      verbs:
          - list
    - apiGroups:
          - trident.netapp.io
      resources:
          - tridentbackends
      verbs:
          - list
          - watch
    - apiGroups:
          - trident.netapp.io
      resources:
          - tridentorchestrators
      verbs:
          - get
          - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
    name: k8s-access-binding
subjects:
    - kind: Group
      name: cloudmanager-access-group
      apiGroup: rbac.authorization.k8s.io
roleRef:
    kind: ClusterRole
    name: cloudmanager-access-clusterrole
    apiGroup: rbac.authorization.k8s.io
      Storage classes

      Add expanded authorization to add storage classes using BlueXP.

      apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
    name: cloudmanager-access-clusterrole
rules:
    - apiGroups:
          - ''
      resources:
          - secrets
          - namespaces
          - persistentvolumeclaims
          - persistentvolumes
          - pods
          - pods/exec
      verbs:
          - get
          - list
          - watch
          - create
          - delete
          - watch
    - apiGroups:
          - storage.k8s.io
      resources:
          - storageclasses
      verbs:
          - get
          - create
          - list
          - watch
          - delete
          - patch
    - apiGroups:
          - trident.netapp.io
      resources:
          - tridentbackends
          - tridentorchestrators
          - tridentbackendconfigs
      verbs:
          - get
          - list
          - watch
          - create
          - delete
          - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
    name: k8s-access-binding
subjects:
    - kind: Group
      name: cloudmanager-access-group
      apiGroup: rbac.authorization.k8s.io
roleRef:
    kind: ClusterRole
    name: cloudmanager-access-clusterrole
    apiGroup: rbac.authorization.k8s.io
      Trident installation

      Use the command line to provide full authorization and enable BlueXP to install Astra Trident.

      eksctl create iamidentitymapping --cluster < > --region < > --arn < > --group "system:masters" --username system:node:{{EC2PrivateDNSName}}

    2. Apply the configuration to a cluster.

      kubectl apply -f <file-name>

  2. Create an identity mapping to the permissions group.

    Use eksctl

    Use eksctl to create an IAM identity mapping between a cluster and the IAM role for the BlueXP Connector.

    Go to the eksctl documentation for full instructions.

    An example is provided below.

    eksctl create iamidentitymapping --cluster <eksCluster> --region <us-east-2> --arn <ARN of the Connector IAM role> --group cloudmanager-access-group --username system:node:{{EC2PrivateDNSName}}
    Edit aws-auth

    Directly edit the aws-auth ConfigMap to add RBAC access to the IAM role for the BlueXP Connector.

    Go to the AWS EKS documentation for full instructions.

    An example is provided below.

    apiVersion: v1
data:
  mapRoles: |
    - groups:
      - cloudmanager-access-group
      rolearn: <ARN of the Connector IAM role>
     username: system:node:{{EC2PrivateDNSName}}
kind: ConfigMap
metadata:
  creationTimestamp: "2021-09-30T21:09:18Z"
  name: aws-auth
  namespace: kube-system
  resourceVersion: "1021"
  selfLink: /api/v1/namespaces/kube-system/configmaps/aws-auth
  uid: dcc31de5-3838-11e8-af26-02e00430057c