简体中文版经机器翻译而成,仅供参考。如与英语版出现任何冲突,应以英语版为准。

8. 部署管理集群

提供者 kevin-hoke

在 Anthos 解决方案中部署的所有 Kubernetes 集群都是从您刚刚创建的 Anthos 管理工作站进行部署的。用户使用 SSH ,上一步创建的公有密钥以及 VM 部署结束时提供的 IP 地址登录到管理工作站。管理集群控制 Anthos 环境中的所有操作。必须先部署管理集群,然后才能根据特定工作负载需求部署各个用户集群。

注 部署使用静态 IP 地址的集群的具体步骤如下 "此处"有关使用 DHCP 的环境的步骤,请参见 。在本指南中,我们使用第二组说明来简化部署。

要部署管理集群,请完成以下步骤:

  1. 使用部署结束时提示的 SSH 命令登录到管理工作站。成功进行身份验证后,您可以列出主目录中的文件,这些文件用于稍后创建管理集群和其他集群。此目录还包括复制的 vCenter 证书以及在先前步骤中创建的 Anthos 的访问密钥。

    [user@rhel7 anthos-install]$ ssh -i ~/.ssh/gke-admin-workstation ubuntu@10.63.172.10
    
    Welcome to Ubuntu 18.04.5 LTS (GNU/Linux 5.4.0-1001-gkeop x86_64)
    
     * Documentation:  https://help.ubuntu.com
     * Management:     https://landscape.canonical.com
     * Support:        https://ubuntu.com/advantage
    Last login: Fri Jan 29 15:46:35 2021 from 10.249.129.216
    
    ubuntu@gke-admin-200915-151421:~$ ls
    admin-cluster.yaml
    user-cluster.yaml
    vcenter.pem
    component-access-key.json
  2. 使用 scp 从部署管理工作站的工作站复制您的 Anthos 帐户的其余密钥。

    ubuntu@gke-admin-200915-151421:~$ scp user@rhel7:~/anthos-install/connect-register-key.json ./
    ubuntu@gke-admin-200915-151421:~$ scp user@rhel7:~/anthos-install/connect-agent-key.json  ./
    ubuntu@gke-admin-200915-151421:~$ scp user@rhel7:~/anthos-install/logging-monitoring-key.json ./
  3. 编辑 admin-cluster.yaml 文件,使其特定于已部署的环境。文件非常大,因此我们将按章节来解决。

    1. 默认情况下,大多数信息都已根据 gkeadm 用于部署管理工作站的配置进行填充。第一部分确认了有关所部署的 Anthos 版本及其所部署的 vCenter 实例的信息。此外,还可以为 Kubernetes 对象数据定义本地数据磁盘( VMDK )。

      apiVersion: v1
      kind: AdminCluster
      # (Required) Absolute path to a GKE bundle on disk
      bundlePath: /var/lib/gke/bundles/gke-onprem-vsphere-1.6.0-gke.7-full.tgz
      # (Required) vCenter configuration
      vCenter:
        address: anthos-vc.cie.netapp.com
        datacenter: NetApp-HCI-Datacenter-01
        cluster: NetApp-HCI-Cluster-01
        resourcePool: Anthos-Resource-Pool
        datastore: VM_Datastore
        # Provide the path to vCenter CA certificate pub key for SSL verification
        caCertPath: "/home/ubuntu/vcenter.pem"
        # The credentials to connect to vCenter
        credentials:
          username: administrator@vsphere.local
          password: "vSphereAdminPassword"
        # Provide the name for the persistent disk to be used by the deployment (ending
        # in .vmdk). Any directory in the supplied path must be created before deployment
        dataDisk: "admin-cluster-disk.vmdk"
    2. 填写下面的网络部分,然后选择使用的是静态模式还是 DHCP 模式。如果使用的是静态地址,则必须根据上述链接的说明创建一个 IP 块文件,并将其添加到配置文件中。

      注 如果在部署中使用静态 IP ,则主机配置下的项为全局项。其中包括集群的静态 IP 或用于 seesaw 负载平衡器的静态 IP ,这些 IP 稍后会进行配置。
      # (Required) Network configuration
      network:
      # (Required) Hostconfig for static addresseses on Seesaw LB's
        hostConfig:
          dnsServers:
          - "10.61.184.251"
          - "10.61.184.252"
          ntpServers:
          - "0.pool.ntp.org"
          - "1.pool.ntp.org"
          - "2.pool.ntp.org"
          searchDomainsForDNS:
          - "cie.netapp.com"
        ipMode:
          # (Required) Define what IP mode to use ("dhcp" or "static")
          type: dhcp
          # # (Required when using "static" mode) The absolute or relative path to the yaml file
          # # to use for static IP allocation
          # ipBlockFilePath: ""
        # (Required) The Kubernetes service CIDR range for the cluster. Must not overlap
        # with the pod CIDR range
        serviceCIDR: 10.96.232.0/24
        # (Required) The Kubernetes pod CIDR range for the cluster. Must not overlap with
        # the service CIDR range
        podCIDR: 192.168.0.0/16
        vCenter:
          # vSphere network name
          networkName: VM_Network
    3. 接下来,填写负载平衡器部分。根据所部署的负载平衡器的类型,此设置可能会有所不同。

      seesaw 示例:

      loadBalancer:
        # (Required) The VIPs to use for load balancing
        vips:
          # Used to connect to the Kubernetes API
          controlPlaneVIP: "10.63.172.155"
          # # (Optional) Used for admin cluster addons (needed for multi cluster features). Must
          # # be the same across clusters
          # # addonsVIP: "10.63.172.153"
        # (Required) Which load balancer to use "F5BigIP" "Seesaw" or "ManualLB". Uncomment
        # the corresponding field below to provide the detailed spec
        kind: Seesaw
        # # (Required when using "ManualLB" kind) Specify pre-defined nodeports
        # manualLB:
        #   # NodePort for ingress service's http (only needed for user cluster)
        #   ingressHTTPNodePort: 0
        #   # NodePort for ingress service's https (only needed for user cluster)
        #   ingressHTTPSNodePort: 0
        #   # NodePort for control plane service
        #   controlPlaneNodePort: 30968
        #   # NodePort for addon service (only needed for admin cluster)
        #   addonsNodePort: 31405
        # # (Required when using "F5BigIP" kind) Specify the already-existing partition and
        # # credentials
        # f5BigIP:
        #   address:
        #   credentials:
        #     username:
        #     password:
        #   partition:
        #   # # (Optional) Specify a pool name if using SNAT
        #   # snatPoolName: ""
        # (Required when using "Seesaw" kind) Specify the Seesaw configs
        seesaw:
        # (Required) The absolute or relative path to the yaml file to use for IP allocation
        #  for LB VMs. Must contain one or two IPs.
        ipBlockFilePath: "admin-seesaw-block.yaml"
        #   (Required) The Virtual Router IDentifier of VRRP for the Seesaw group. Must
        #   be between 1-255 and unique in a VLAN.
          vrid: 100
        #   (Required) The IP announced by the master of Seesaw group
          masterIP: "10.63.172.151"
        #   (Required) The number CPUs per machine
          cpus: 1
        #   (Required) Memory size in MB per machine
          memoryMB: 2048
        #   (Optional) Network that the LB interface of Seesaw runs in (default: cluster
        #   network)
          vCenter:
        #   vSphere network name
            networkName: VM_Network
        #   (Optional) Run two LB VMs to achieve high availability (default: false)
          enableHA: false
    4. 对于 seesaw 负载平衡器,您必须创建一个额外的外部文件来为负载平衡器提供静态 IP 信息。创建文件 admin-seesaw-block.yaml ,此文件已在本配置部分中引用。

      blocks:
        - netmask: "255.255.255.0"
          gateway: "10.63.172.1"
          ips:
          - ip: "10.63.172.152"
            hostname: "admin-seesaw-vm"

      F5 BigIP 示例:

      # (Required) Load balancer configuration
      loadBalancer:
        # (Required) The VIPs to use for load balancing
        vips:
          # Used to connect to the Kubernetes API
          controlPlaneVIP: "10.63.172.155"
          # # (Optional) Used for admin cluster addons (needed for multi cluster features). Must
          # # be the same across clusters
          # # addonsVIP: "10.63.172.153"
        # (Required) Which load balancer to use "F5BigIP" "Seesaw" or "ManualLB". Uncomment
        # the corresponding field below to provide the detailed spec
        kind: F5BigIP
        # # (Required when using "ManualLB" kind) Specify pre-defined nodeports
        # manualLB:
        #   # NodePort for ingress service's http (only needed for user cluster)
        #   ingressHTTPNodePort: 0
        #   # NodePort for ingress service's https (only needed for user cluster)
        #   ingressHTTPSNodePort: 0
        #   # NodePort for control plane service
        #   controlPlaneNodePort: 30968
        #   # NodePort for addon service (only needed for admin cluster)
        #   addonsNodePort: 31405
        # # (Required when using "F5BigIP" kind) Specify the already-existing partition and
        # # credentials
        f5BigIP:
          address: "172.21.224.21"
          credentials:
            username: "admin"
            password: "admin-password"
          partition: "Admin-Cluster"
        #   # # (Optional) Specify a pool name if using SNAT
        #   # snatPoolName: ""
        # (Required when using "Seesaw" kind) Specify the Seesaw configs
        # seesaw:
          # (Required) The absolute or relative path to the yaml file to use for IP allocation
          # for LB VMs. Must contain one or two IPs.
          #  ipBlockFilePath: ""
          # (Required) The Virtual Router IDentifier of VRRP for the Seesaw group. Must
          # be between 1-255 and unique in a VLAN.
          #  vrid: 0
          # (Required) The IP announced by the master of Seesaw group
          #  masterIP: ""
          # (Required) The number CPUs per machine
          #  cpus: 4
          # (Required) Memory size in MB per machine
          #   memoryMB: 8192
          # (Optional) Network that the LB interface of Seesaw runs in (default: cluster
          # network)
          #   vCenter:
            # vSphere network name
            #     networkName: VM_Network
          # (Optional) Run two LB VMs to achieve high availability (default: false)
          #   enableHA: false
    5. 管理员配置文件的最后一部分包含可根据特定部署环境进行调整的其他选项。其中包括,如果在少于三个 ESXi 服务器上部署 Anthos ,则启用反关联性组。您还可以配置代理,专用 Docker 注册表以及与 Stackdriver 和 Google Cloud 的连接以进行审核。

      antiAffinityGroups:
        # Set to false to disable DRS rule creation
        enabled: false
      # (Optional) Specify the proxy configuration
      proxy:
        # The URL of the proxy
        url: ""
        # The domains and IP addresses excluded from proxying
        noProxy: ""
      # # (Optional) Use a private Docker registry to host GKE images
      # privateRegistry:
      #   # Do not include the scheme with your registry address
      #   address: ""
      #   credentials:
      #     username: ""
      #     password: ""
      #   # The absolute or relative path to the CA certificate for this registry
      #   caCertPath: ""
      # (Required): The absolute or relative path to the GCP service account key for pulling
      # GKE images
      gcrKeyPath: "/home/ubuntu/component-access-key.json"
      # (Optional) Specify which GCP project to connect your logs and metrics to
      stackdriver:
        projectID: "anthos-dev"
        # A GCP region where you would like to store logs and metrics for this cluster.
        clusterLocation: "us-east1"
        enableVPC: false
        # The absolute or relative path to the key file for a GCP service account used to
        # send logs and metrics from the cluster
        serviceAccountKeyPath: "/home/ubuntu/logging-monitoring-key.json"
      # # (Optional) Configure kubernetes apiserver audit logging
      # cloudAuditLogging:
      #   projectid: ""
      #   # A GCP region where you would like to store audit logs for this cluster.
      #   clusterlocation: ""
      #   # The absolute or relative path to the key file for a GCP service account used to
      #   # send audit logs from the cluster
      #   serviceaccountkeypath: ""
      注 本文档中详细介绍的部署是一种最低配置,用于验证,需要禁用反关联性规则。NetApp 建议在生产部署中将此选项设置为 true 。
    注 默认情况下, VMware 上的 Anthos 使用预先存在的 Google 拥有的容器映像注册表,无需进行其他设置。如果您选择使用私有 Docker 注册表进行部署,则必须根据找到的说明单独配置该注册表 "此处"。本部署指南不会介绍此步骤。
  4. 对 admin-cluster.yaml 文件的编辑完成后,请务必检查语法和间距是否正确。

    ubuntu@gke-admin-200915-151421:~$ gkectl check-config –config admin-cluster.yaml
  5. 通过配置检查并解决任何已确定的问题后,您可以暂存集群部署。由于我们已经检查了配置文件的验证,因此可以通过传递 ` – -skip-validation-all` 标志来跳过这些步骤。

    ubuntu@gke-admin-200915-151421:~$ gkectl prepare --config admin-cluster.yaml --skip-validation-all
  6. 如果您使用的是 seesaw 负载平衡器,则必须在部署集群本身之前创建一个负载平衡器(否则,请跳过此步骤)。

    ubuntu@gke-admin-200915-151421:~$ gkectl create loadbalancer --config admin-cluster.yaml
  7. 现在,您可以建立管理集群。可以使用 gkectl create admin 命令来完成此操作,该命令可以使用 ` – -skip-validation-all` 标志来加快部署速度。

    ubuntu@gke-admin-200915-151421:~$ gkectl create admin --config admin-cluster.yaml --skip-validation-all
  8. 部署集群时,它会在本地目录中创建 kubeconfig 文件。可以使用此文件使用 kubectl 检查集群状态,或者使用 gkectl 运行诊断。

    ubuntu@gke-admin-ws-200915-151421:~ $ kubectl get nodes --kubeconfig kubeconfig
    NAME                                     STATUS   ROLES    AGE    VERSION
    gke-admin-master-gkvmp                   Ready    master   5m    v1.18.6-gke.6600
    gke-admin-node-84b77ff5c7-6zg59          Ready    <none>   5m    v1.18.6-gke.6600
    gke-admin-node-84b77ff5c7-8jdmz          Ready    <none>   5m    v1.18.6-gke.6600
    ubuntu@gke-admin-ws-200915-151421:~$ gkectl diagnose cluster –-kubeconfig kubeconfig
    Diagnosing admin cluster "gke-admin-gkvmp"...- Validation Category: Admin Cluster VCenter
    Checking Credentials...SUCCESS
    Checking Version...SUCCESS
    Checking Datacenter...SUCCESS
    Checking Datastore...SUCCESS
    Checking Resource pool...SUCCESS
    Checking Folder...SUCCESS
    Checking Network...SUCCESS- Validation Category: Admin Cluster
    Checking cluster object...SUCCESS
    Checking machine deployment...SUCCESS
    Checking machineset...SUCCESS
    Checking machine objects...SUCCESS
    Checking kube-system pods...SUCCESS
    Checking storage...SUCCESS
    Checking resource...System pods on UserMaster cpu resource request report: total 1754m nodeCount 2 min 877m max 877m avg 877m tracked amount in bundle 4000m
    System pods on AdminNode cpu resource request report: total 2769m nodeCount 2 min 1252m max 1517m avg 1384m tracked amount in bundle 4000m
    System pods on AdminMaster cpu resource request report: total 923m nodeCount 1 min 923m max 923m avg 923m tracked amount in bundle 4000m
    System pods on UserMaster memory resource request report: total 4524461824 nodeCount 2 min 2262230912 max 2262230912 avg 2262230912 tracked amount in bundle 8192Mi
    System pods on AdminNode memory resource request report: total 6876Mi nodeCount 2 min 2174Mi max 4702Mi avg 3438Mi tracked amount in bundle 16384Mi
    System pods on AdminMaster memory resource request report: total 465Mi nodeCount 1 min 465Mi max 465Mi avg 465Mi tracked amount in bundle 16384Mi
    SUCCESS
    Cluster is healthy.