简体中文版经机器翻译而成,仅供参考。如与英语版出现任何冲突,应以英语版为准。

部署用户集群

提供者

借助 Anthos ,企业可以扩展其环境,以整合多个用户集群并在各个团队之间隔离工作负载。一个管理集群最多可支持 20 个用户集群,每个用户集群最多可支持 250 个节点和 7500 个 Pod 。

要为您的部署配置用户集群,请完成以下步骤:

  1. 在部署 anthos-admin 工作站时,会创建一个名为 user-cluster.yaml 的文件,该文件可用于为正在运行的工作负载部署多个额外的用户集群。首先,使用要部署的每个集群的新名称复制此默认文件。

    ubuntu@gke-admin-ws-200915-151421:~ $ cp config.yaml anthos-cluster01-config.yaml
  2. 编辑 anthos-cluster01-config.yaml 文件,使其特定于要部署的环境。

    1. 与先前使用的 admin-config.yaml 类似,大多数变量都已填充,或者它们会引用管理集群来获取部署所需的信息。第一部分确认了有关所部署的 Anthos 版本及其所部署的 vCenter 实例的信息。

      apiVersion: v1
      kind: UserCluster
      # (Required) A unique name for this cluster
      name: "anthos-cluster01"
      # (Required) GKE on-prem version (example: 1.3.0-gke.16)
      gkeOnPremVersion: 1.6.0-gke.7
      # # (Optional) vCenter configuration (default: inherit from the admin cluster)
      # vCenter:
      #   resourcePool: ""
      #   datastore: ""
      #   # Provide the path to vCenter CA certificate pub key for SSL verification
      #   caCertPath: ""
      #   # The credentials to connect to vCenter
      #   credentials:
      #     username: ""
      #     password: ""
    2. 您必须填写下面的网络连接部分,然后选择使用的是静态模式还是 DHCP 模式。如果使用的是静态地址,则必须创建一个 IP 块文件以提供类似于管理集群配置的地址。

      注 任何时候在部署中使用静态 IP 时, hostConfig 部分下的项都是全局的。这包括集群的静态 IP 以及用于 seesaw 负载平衡器的静态 IP ,这些 IP 稍后会进行配置。
      # (Required) Network configuration; vCenter section is optional and inherits from
      # the admin cluster if not specified
      network:
      # (Required) Hostconfig for static addresseses on Seesaw LB's
        hostConfig:
          dnsServers:
          - "10.61.184.251"
          - "10.61.184.252"
          ntpServers:
          - "0.pool.ntp.org"
          - "1.pool.ntp.org"
          - "2.pool.ntp.org"
          searchDomainsForDNS:
          - "cie.netapp.com"
        ipMode:
          # (Required) Define what IP mode to use ("dhcp" or "static")
          type: dhcp
          # # (Required when using "static" mode) The absolute or relative path to the yaml file
          # # to use for static IP allocation
          # ipBlockFilePath: ""
        # (Required) The Kubernetes service CIDR range for the cluster. Must not overlap
        # with the pod CIDR range
        serviceCIDR: 10.96.0.0/12
        # (Required) The Kubernetes pod CIDR range for the cluster. Must not overlap with
        # the service CIDR range
        podCIDR: 192.168.0.0/16
        vCenter:
          # vSphere network name
          networkName: VM_Network
    3. 接下来,填写负载平衡器部分。根据所部署的负载平衡器的类型,此设置可能会有所不同。

      seesaw 示例:

      # (Required) Load balancer configuration
      loadBalancer:
        # (Required) The VIPs to use for load balancing
        vips:
          # Used to connect to the Kubernetes API
          controlPlaneVIP: "10.63.172.156"
          # Shared by all services for ingress traffic
          ingressVIP: "10.63.172.157"
        # (Required) Which load balancer to use "F5BigIP" "Seesaw" or "ManualLB". Uncomment
        # the corresponding field below to provide the detailed spec
        kind: Seesaw
        # # (Required when using "ManualLB" kind) Specify pre-defined nodeports
        # manualLB:
        #   # NodePort for ingress service's http (only needed for user cluster)
        #   ingressHTTPNodePort: 30243
        #   # NodePort for ingress service's https (only needed for user cluster)
        #   ingressHTTPSNodePort: 30879
        #   # NodePort for control plane service
        #   controlPlaneNodePort: 30562
        #   # NodePort for addon service (only needed for admin cluster)
        #   addonsNodePort: 0
        # # (Required when using "F5BigIP" kind) Specify the already-existing partition and
        # # credentials
      # f5BigIP:
        #  address:
        #  credentials:
        #    username:
        #    password:
        #  partition:
        #   # # (Optional) Specify a pool name if using SNAT
        #   # snatPoolName: ""
        # (Required when using "Seesaw" kind) Specify the Seesaw configs
        seesaw:
          # (Required) The absolute or relative path to the yaml file to use for IP allocation
          # for LB VMs. Must contain one or two IPs.
          ipBlockFilePath: "anthos-cluster01-seesaw-block.yaml"
          # (Required) The Virtual Router IDentifier of VRRP for the Seesaw group. Must
          # be between 1-255 and unique in a VLAN.
          vrid: 101
          # (Required) The IP announced by the master of Seesaw group
          masterIP: "10.63.172.153"
          # (Required) The number CPUs per machine
          cpus: 1
          # (Required) Memory size in MB per machine
          memoryMB: 2048
          # (Optional) Network that the LB interface of Seesaw runs in (default: cluster
          # network)
          vCenter:
          # vSphere network name
            networkName: VM_Network
          # (Optional) Run two LB VMs to achieve high availability (default: false)
          enableHA: false
    4. 对于 seesaw 负载平衡器,您必须创建一个额外的外部文件来为负载平衡器提供静态 IP 信息。创建此配置部分中引用的文件 anthos-cluster01-seesaw-block.yaml

      blocks:
        - netmask: "255.255.255.0"
          gateway: "10.63.172.1"
          ips:
          - ip: "10.63.172.154"
            hostname: "anthos-cluster01-seesaw-vm"

      F5 BigIP 示例:

    loadBalancer:
      # (Required) The VIPs to use for load balancing
      vips:
        # Used to connect to the Kubernetes API
        controlPlaneVIP: "10.63.172.158"
        # Shared by all services for ingress traffic
        ingressVIP: "10.63.172.159"
      # (Required) Which load balancer to use "F5BigIP" "Seesaw" or "ManualLB". Uncomment
      # the corresponding field below to provide the detailed spec
      kind: F5BigIP
      # # (Required when using "ManualLB" kind) Specify pre-defined nodeports
      # manualLB:
      #   # NodePort for ingress service's http (only needed for user cluster)
      #   ingressHTTPNodePort: 30243
      #   # NodePort for ingress service's https (only needed for user cluster)
      #   ingressHTTPSNodePort: 30879
      #   # NodePort for control plane service
      #   controlPlaneNodePort: 30562
      #   # NodePort for addon service (only needed for admin cluster)
      #   addonsNodePort: 0
      # # (Required when using "F5BigIP" kind) Specify the already-existing partition and
      # # credentials
      f5BigIP:
        address: "172.21.224.21"
        credentials:
          username: "admin"
          password: "admin-password"
        partition: "Anthos-Cluster-01"
      #   # # (Optional) Specify a pool name if using SNAT
      #   # snatPoolName: ""
      # (Required when using "Seesaw" kind) Specify the Seesaw configs
      # seesaw:
        # (Required) The absolute or relative path to the yaml file to use for IP allocation
        # for LB VMs. Must contain one or two IPs.
        #  ipBlockFilePath: ""
        # (Required) The Virtual Router IDentifier of VRRP for the Seesaw group. Must
        # be between 1-255 and unique in a VLAN.
        #  vrid: 0
        # (Required) The IP announced by the master of Seesaw group
        #  masterIP: ""
        # (Required) The number CPUs per machine
        #  cpus: 4
        # (Required) Memory size in MB per machine
        #   memoryMB: 8192
        # (Optional) Network that the LB interface of Seesaw runs in (default: cluster
        # network)
        #   vCenter:
          # vSphere network name
          #     networkName: VM_Network
        # (Optional) Run two LB VMs to achieve high availability (default: false)
        #   enableHA: false
    1. 最后一节介绍了集群要部署的节点的资源,包括创建一个节点池,供我们稍后用于动态扩展。本节还会提供服务帐户密钥,以便在部署 GKE之后 向其注册集群。

      # (Optional) User cluster master nodes must have either 1 or 3 replicas (default:
      # 4 CPUs; 16384 MB memory; 1 replica)
      masterNode:
        cpus: 4
        memoryMB: 8192
        # How many machines of this type to deploy
        replicas: 1
      # (Required) List of node pools. The total un-tainted replicas across all node pools
      # must be greater than or equal to 3
      nodePools:
      - name: anthos-cluster01
        # # Labels to apply to Kubernetes Node objects
        # labels: {}
        # # Taints to apply to Kubernetes Node objects
        # taints:
        # - key: ""
        #   value: ""
        #   effect: ""
        cpus: 4
        memoryMB: 8192
        # How many machines of this type to deploy
        replicas: 3
      # Spread nodes across at least three physical hosts (requires at least three hosts)
      antiAffinityGroups:
        # Set to false to disable DRS rule creation
        enabled: false
      # # (Optional): Configure additional authentication
      # authentication:
      #   # (Optional) Configure OIDC authentication
      #   oidc:
      #     issuerURL: ""
      #     kubectlRedirectURL: ""
      #     clientID: ""
      #     clientSecret: ""
      #     username: ""
      #     usernamePrefix: ""
      #     group: ""
      #     groupPrefix: ""
      #     scopes: ""
      #     extraParams: ""
      #     # Set value to string "true" or "false"
      #     deployCloudConsoleProxy: ""
      #     # # The absolute or relative path to the CA file (optional)
      #     # caPath: ""
      #   # (Optional) Provide an additional serving certificate for the API server
      #   sni:
      #     certPath: ""
      #     keyPath: ""
      #   # (Optional) Configure LDAP authentication (preview feature)
      #   ldap:
      #     name: ""
      #     host: ""
      #     # Only support "insecure" for now (optional)
      #     connectionType: insecure
      #     # # The absolute or relative path to the CA file (optional)
      #     # caPath: ""
      #     user:
      #       baseDN: ""
      #       userAttribute: ""
      #       memberAttribute: ""
      # (Optional) Specify which GCP project to connect your logs and metrics to
      stackdriver:
        projectID: "anthos-dev"
        # A GCP region where you would like to store logs and metrics for this cluster.
        clusterLocation: "us-east1"
        enableVPC: false
        # The absolute or relative path to the key file for a GCP service account used to
        # send logs and metrics from the cluster
        serviceAccountKeyPath: "/home/ubuntu/logging-monitoring-key.json "
      # (Optional) Specify which GCP project to connect your GKE clusters to
      gkeConnect:
        projectID: "anthos-dev"
        # The absolute or relative path to the key file for a GCP service account used to
        # register the cluster
        registerServiceAccountKeyPath: "/home/ubuntu/connect-register-key.json"
        # The absolute or relative path to the key file for a GCP service account used by
        # the GKE connect agent
        agentServiceAccountKeyPath: "/home/ubuntu/component-access-key.json"
      # (Optional) Specify Cloud Run configuration
      cloudRun:
        enabled: false
      # # (Optional/Alpha) Configure the GKE usage metering feature
      # usageMetering:
      #   bigQueryProjectID: ""
      #   # The ID of the BigQuery Dataset in which the usage metering data will be stored
      #   bigQueryDatasetID: ""
      #   # The absolute or relative path to the key file for a GCP service account used by
      #   # gke-usage-metering to report to BigQuery
      #   bigQueryServiceAccountKeyPath: ""
      #   # Whether or not to enable consumption-based metering
      #   enableConsumptionMetering: false
      # # (Optional/Alpha) Configure kubernetes apiserver audit logging
      # cloudAuditLogging:
      #   projectid: ""
      #   # A GCP region where you would like to store audit logs for this cluster.
      #   clusterlocation: ""
      #   # The absolute or relative path to the key file for a GCP service account used to
      #   # send audit logs from the cluster
      #   serviceaccountkeypath: ""
  3. 对配置文件的编辑完成后, NetApp 建议检查该文件的语法和间距是否正确。您可以检查刚刚创建的配置文件。此命令会引用管理集群创建的 kubeconfig 文件。

    ubuntu@gke-admin-200915-151421:~$ gkectl check-config --kubeconfig kubeconfig --config anthos-cluster01-config.yaml
  4. 如果您使用的是 seesaw 负载平衡器,则需要在部署用户集群之前创建它。

    ubuntu@gke-admin-200915-151421:~$ gkectl create loadbalancer -–kubeconfig kubeconfig –-config anthos-cluster-01-config.yaml
  5. 创建用户集群。与管理集群一样,可以跳过其他验证来加快此过程,因为我们已经在上一步中运行了检查。

    ubuntu@gke-admin-200915-151421:~$ gkectl create cluster –-config anthos-cluster-01-config.yaml –-skip-validation-all
  6. 部署集群时,它会在本地目录中创建 kubeconfig 文件。此文件可用于使用 kubectl 检查集群状态,或者用于使用 gkectl 运行诊断。

    ubuntu@gke-admin-ws-200915-151421:~$ kubectl get nodes --kubeconfig anthos-cluster01-kubeconfig
    NAME                    STATUS   ROLES    AGE   VERSION
    anthos-cluster01-7b5995cc45-ftrdw   Ready    <none>   5m   v1.18.6-gke.6600
    anthos-cluster01-7b5995cc45-z7q9b   Ready    <none>   5m   v1.18.6-gke.6600
    anthos-cluster01-7b5995cc45-zw6sv   Ready    <none>   6m   v1.18.6-gke.6600
    ubuntu@gke-admin-ws-200915-151421:~/ $ gkectl diagnose cluster --kubeconfig kubeconfig --cluster-name anthos-cluster01
    Diagnosing user cluster "anthos-cluster01"...
    
    - Validation Category: User Cluster VCenter
    Checking Credentials...SUCCESS
    Checking VSphere CSI Driver...SUCCESS
    Checking Version...SUCCESS
    Checking Datacenter...SUCCESS
    Checking Datastore...SUCCESS
    Checking Resource pool...SUCCESS
    Checking Folder...SUCCESS
    Checking Network...SUCCESS
    Checking Datastore...SUCCESS
    
    - Validation Category: User Cluster
    Checking onpremusercluster and onpremnodepool...SUCCESS
    Checking cluster object...SUCCESS
    Checking machine deployment...SUCCESS
    Checking machineset...SUCCESS
    Checking machine objects...SUCCESS
    Checking control place pods...SUCCESS
    Checking gke-connect pods...SUCCESS
    Checking config-management-system pods...Warning: No pod is running in namespace "config-management-system"...SUCCESS
    Checking kube-system pods...SUCCESS
    Checking gke-system pods...SUCCESS
    Checking storage...SUCCESS
    Checking resource...System pods on UserNode cpu resource request report: total 3059m nodeCount 3 min 637m max 1224m avg 1019m tracked amount in bundle 4000m
    System pods on UserNode memory resource request report: total 6464Mi nodeCount 3 min 1670Mi max 2945Mi avg 2259331754 tracked amount in bundle 8192Mi
    SUCCESS
    Cluster is healthy.