简体中文版经机器翻译而成,仅供参考。如与英语版出现任何冲突,应以英语版为准。

配置网络交换机(自动部署)

提供者 kevin-hoke

准备所需的 VLAN ID

下表列出了部署所需的 VLAN ,如此解决方案验证中所述。在执行 NDE 之前,您应在网络交换机上配置这些 VLAN 。

网段 详细信息 VLAN ID

带外管理网络

HCI 终端用户界面( TUI )网络

16.

带内管理网络

用于访问节点,主机和子系统的管理接口的网络

3488

VMware vMotion

用于实时迁移虚拟机的网络

3489

iSCSI SAN 存储

用于 iSCSI 存储流量的网络

3490

应用程序

应用程序流量网络

3487

NFS

NFS 存储流量的网络

3491.

IPL*

Mellanox 交换机之间的对等链路

4000

原生

原生 VLAN

2.

  • 仅适用于 Mellanox 交换机

交换机配置

此解决方案使用运行 Onyx 的 Mellanox SN2010 交换机。Mellanox 交换机使用 Ansible 攻略手册进行配置。在运行 Ansible 攻略手册之前,您应手动执行交换机的初始配置:

  1. 将交换机安装到上行链路交换机,计算和存储节点并通过缆线连接。

  2. 打开交换机的电源,并使用以下详细信息对其进行配置:

    1. 主机名

    2. 管理 IP 和网关

    3. NTP

  3. 登录到 Mellanox 交换机并运行以下命令:

    configuration write to pre-ansible
    configuration write to post-ansible

    创建的 Pre-Ansible 配置文件可用于将交换机的配置还原到 Ansible 攻略手册执行前的状态。

    此解决方案的交换机配置存储在 POST/Ansible 配置文件中。

  4. 可以从下载符合 NetApp HCI 最佳实践和要求的 Mellanox 交换机配置攻略手册 "NetApp HCI 工具包"

    注 HCI 工具包还提供了一本攻略手册,用于为 Cisco Nexus 交换机设置与 NetApp HCI 类似的最佳实践和要求。
    注 有关填充变量和执行攻略手册的其他指导,请参见相应的交换机 README.md 文件。
  5. 填写凭据以访问环境所需的交换机和变量。以下文本是此解决方案的变量文件示例。

    # vars file for nar_hci_mellanox_deploy
    #These set of variables will setup the Mellanox switches for NetApp HCI that uses a 2-cable compute connectivity option.
    #Ansible connection variables for mellanox
    ansible_connection: network_cli
    ansible_network_os: onyx
    #--------------------
    # Primary Variables
    #--------------------
    #Necessary VLANs for Standard NetApp HCI Deployment [native, Management, iSCSI_Storage, vMotion, VM_Network, IPL]
    #Any additional VLANs can be added to this in the prescribed format below
    netapp_hci_vlans:
    - {vlan_id: 2 , vlan_name: "Native" }
    - {vlan_id: 3488 , vlan_name: "IB-Management" }
    - {vlan_id: 3490 , vlan_name: "iSCSI_Storage" }
    - {vlan_id: 3489 , vlan_name: "vMotion" }
    - {vlan_id: 3491 , vlan_name: "NFS " }
    - {vlan_id: 3487 , vlan_name: "App_Network" }
    - {vlan_id: 4000 , vlan_name: "IPL" }#Modify the VLAN IDs to suit your environment
    #Spanning-tree protocol type for uplink connections.
    #The valid options are 'network' and 'normal'; selection depends on the uplink switch model.
    uplink_stp_type: network
    #----------------------
    # IPL variables
    #----------------------
    #Inter-Peer Link Portchannel
    #ipl_portchannel to be defined in the format - Po100
    ipl_portchannel: Po100
    #Inter-Peer Link Addresses
    #The IPL IP address should not be part of the management network. This is typically a private network
    ipl_ipaddr_a: 10.0.0.1
    ipl_ipaddr_b: 10.0.0.2
    #Define the subnet mask in CIDR number format. Eg: For subnet /22, use ipl_ip_subnet: 22
    ipl_ip_subnet: 24
    #Inter-Peer Link Interfaces
    #members to be defined with Eth in the format. Eg: Eth1/1
    peer_link_interfaces:
      members: ['Eth1/20', 'Eth1/22']
      description: "peer link interfaces"
    #MLAG VIP IP address should be in the same subnet as that of the switches' mgmt0 interface subnet
    #mlag_vip_ip to be defined in the format - <vip_ip>/<subnet_mask>. Eg: x.x.x.x/y
    mlag_vip_ip: <<mlag_vip_ip>>
    #MLAG VIP Domain Name
    #The mlag domain must be unique name for each mlag domain.
    #In case you have more than one pair of MLAG switches on the same network, each domain (consist of two switches) should be configured with different name.
    mlag_domain_name: MLAG-VIP-DOM
    #---------------------
    # Interface Details
    #---------------------
    #Storage Bond10G Interface details
    #members to be defined with Eth in the format. Eg: Eth1/1
    #Only numerical digits between 100 to 1000 allowed for mlag_id
    #Operational link speed [variable 'speed' below] to be defined in terms of bytes.
    #For 10 Gigabyte operational speed, define 10G. [Possible values - 10G and 25G]
    #Interface descriptions append storage node data port numbers assuming all Storage Nodes' Port C -> Mellanox Switch A and all Storage Nodes' Port D -> Mellanox Switch B
    #List the storage Bond10G interfaces, their description, speed and MLAG IDs in list of dictionaries format
    storage_interfaces:
    - {members: "Eth1/1", description: "HCI_Storage_Node_01", mlag_id: 101, speed: 25G}
    - {members: "Eth1/2", description: "HCI_Storage_Node_02", mlag_id: 102, speed: 25G}
    #In case of additional storage nodes, add them here
    #Storage Bond1G Interface
    #Mention whether or not these Mellanox switches will also be used for Storage Node Mgmt connections
    #Possible inputs for storage_mgmt are 'yes' and 'no'
    storage_mgmt: <<yes or no>>
    #Storage Bond1G (Mgmt) interface details. Only if 'storage_mgmt' is set to 'yes'
    #Members to be defined with Eth in the format. Eg: Eth1/1
    #Interface descriptions append storage node management port numbers assuming all Storage Nodes' Port A -> Mellanox Switch A and all Storage Nodes' Port B -> Mellanox Switch B
    #List the storage Bond1G interfaces and their description in list of dictionaries format
    storage_mgmt_interfaces:
    - {members: "Ethx/y", description: "HCI_Storage_Node_01"}
    - {members: "Ethx/y", description: "HCI_Storage_Node_02"}
    #In case of additional storage nodes, add them here
    #LACP load balancing algorithm for IP hash method
    #Possible options are: 'destination-mac', 'destination-ip', 'destination-port', 'source-mac', 'source-ip', 'source-port', 'source-destination-mac', 'source-destination-ip', 'source-destination-port'
    #This variable takes multiple options in a single go
    #For eg: if you want to configure load to be distributed in the port-channel based on the traffic source and destination IP address and port number, use 'source-destination-ip source-destination-port'
    #By default, Mellanox sets it to source-destination-mac. Enter the values below only if you intend to configure any other load balancing algorithm
    #Make sure the load balancing algorithm that is set here is also replicated on the host side
    #Recommended algorithm is source-destination-ip source-destination-port
    #Fill the lacp_load_balance variable only if you are using configuring interfaces on compute nodes in bond or LAG with LACP
    lacp_load_balance: "source-destination-ip source-destination-port"
    #Compute Interface details
    #Members to be defined with Eth in the format. Eg: Eth1/1
    #Fill the mlag_id field only if you intend to configure interfaces of compute nodes into bond or LAG with LACP
    #In case you do not intend to configure LACP on interfaces of compute nodes, either leave the mlag_id field unfilled or comment it or enter NA in the mlag_id field
    #In case you have a mixed architecture where some compute nodes require LACP and some don't,
    #1. Fill the mlag_id field with appropriate MLAG ID for interfaces that connect to compute nodes requiring LACP
    #2. Either fill NA or leave the mlag_id field blank or comment it for interfaces connecting to compute nodes that do not require LACP
    #Only numerical digits between 100 to 1000 allowed for mlag_id.
    #Operational link speed [variable 'speed' below] to be defined in terms of bytes.
    #For 10 Gigabyte operational speed, define 10G. [Possible values - 10G and 25G]
    #Interface descriptions append compute node port numbers assuming all Compute Nodes' Port D -> Mellanox Switch A and all Compute Nodes' Port E -> Mellanox Switch B
    #List the compute interfaces, their speed, MLAG IDs and their description in list of dictionaries format
    compute_interfaces:
    - members: "Eth1/7"#Compute Node for ESXi, setup by NDE
      description: "HCI_Compute_Node_01"
      mlag_id: #Fill the mlag_id only if you wish to use LACP on interfaces towards compute nodes
      speed: 25G
    - members: "Eth1/8"#Compute Node for ESXi, setup by NDE
      description: "HCI_Compute_Node_02"
      mlag_id: #Fill the mlag_id only if you wish to use LACP on interfaces towards compute nodes
      speed: 25G
    #In case of additional compute nodes, add them here in the same format as above- members: "Eth1/9"#Compute Node for Kubernetes Worker node
      description: "HCI_Compute_Node_01"
      mlag_id: 109 #Fill the mlag_id only if you wish to use LACP on interfaces towards compute nodes
      speed: 10G
    - members: "Eth1/10"#Compute Node for Kubernetes Worker node
      description: "HCI_Compute_Node_02"
      mlag_id: 110 #Fill the mlag_id only if you wish to use LACP on interfaces towards compute nodes
      speed: 10G
    #Uplink Switch LACP support
    #Possible options are 'yes' and 'no' - Set to 'yes' only if your uplink switch supports LACP
    uplink_switch_lacp: <<yes or no>>
    #Uplink Interface details
    #Members to be defined with Eth in the format. Eg: Eth1/1
    #Only numerical digits between 100 to 1000 allowed for mlag_id.
    #Operational link speed [variable 'speed' below] to be defined in terms of bytes.
    #For 10 Gigabyte operational speed, define 10G. [Possible values in Mellanox are 1G, 10G and 25G]
    #List the uplink interfaces, their description, MLAG IDs and their speed in list of dictionaries format
    uplink_interfaces:
    - members: "Eth1/18"
      description_switch_a: "SwitchA:Ethx/y -> Uplink_Switch:Ethx/y"
      description_switch_b: "SwitchB:Ethx/y -> Uplink_Switch:Ethx/y"
      mlag_id: 118  #Fill the mlag_id only if 'uplink_switch_lacp' is set to 'yes'
      speed: 10G
      mtu: 1500
    注 交换机密钥的指纹必须与执行攻略手册的主机中的指纹一致。要确保这一点,请将此密钥添加到 ` /root/ 。SSH/known_host` 或任何其他适当位置。

回滚交换机配置

  1. 如果发生任何超时故障或部分配置,请运行以下命令将交换机回滚到初始状态。

    configuration switch-to pre-ansible
    注 此操作需要重新启动交换机。
  2. 在运行 Ansible 攻略手册之前,将配置切换为状态。

    configuration delete post-ansible
  3. 从 Ansible 攻略手册中删除具有此配置的后 Ansible 文件。

    configuration write to post-ansible
  4. 创建一个名称与 POST/Ansible 相同的新文件,将 Pre-Ansible 配置写入该文件,然后切换到新配置以重新启动配置。

IP 地址要求

在 VMware 和 Kubernetes 中部署 NetApp HCI 推理平台需要分配多个 IP 地址。下表列出了所需的 IP 地址数量。除非另有说明,否则 NDE 会自动分配地址。

IP 地址数量 详细信息 VLAN ID IP 地址

每个存储和计算节点一个 *

HCI 终端用户界面( TUI )地址

16.

每个 vCenter Server ( VM )一个

vCenter Server 管理地址

3488

每个管理节点( VM )一个

管理节点 IP 地址

每个 ESXi 主机一个

ESXi 计算管理地址

每个存储 / 见证节点一个

NetApp HCI 存储节点管理地址

每个存储集群一个

存储集群管理地址

每个 ESXi 主机一个

VMware vMotion 地址

3489

每个 ESXi 主机两个

iSCSI 存储流量的 ESXi 主机启动程序地址

3490

每个存储节点两个

iSCSI 存储流量的存储节点目标地址

每个存储集群两个

iSCSI 存储流量的存储集群目标地址

两个用于 mNode

mNode iSCSI 存储访问

配置相应组件时,系统会手动分配以下 IP 。

IP 地址数量 详细信息 VLAN ID IP 地址

一个用于部署跳转管理网络

部署跳转 VM 以执行 Ansible 攻略手册并配置系统的其他部分—管理连接

3488

每个 Kubernetes 主节点一个管理网络

Kubernetes 主节点 VM (三个节点)

3488

每个 Kubernetes 工作节点一个管理网络

Kubernetes 工作节点(两个节点)

3488

每个 Kubernetes 工作节点一个— NFS 网络

Kubernetes 工作节点(两个节点)

3491.

每个 Kubernetes 工作节点一个—应用程序网络

Kubernetes 工作节点(两个节点)

3487

三个用于 ONTAP Select 管理网络

ONTAP Select 虚拟机

3488

一个用于 ONTAP Select — NFS 网络

ONTAP Select VM — NFS 数据流量

3491.

至少两个用于 Triton 推理服务器负载平衡器—应用程序网络

Kubernetes 负载平衡器服务的负载平衡器 IP 范围

3487

  • 此验证需要初始设置第一个存储节点 TUI 地址。NDE 会自动为后续节点分配 TUI 地址。

DNS 和计时要求

根据您的部署,您可能需要为 NetApp HCI 系统准备 DNS 记录。NetApp HCI 需要一个有效的 NTP 服务器来进行计时;如果您的环境中没有可用的时间服务器,则可以使用一个公共时间服务器。

此验证涉及使用完全限定域名( FQDN )为新的 VMware vCenter Server 实例部署 NetApp HCI 。在部署之前,必须在 DNS 服务器上创建一个指针( PTR )记录和一个地址( A )记录。