Skip to main content
NetApp HCI Solutions

2. Configure Data Switches: NetApp HCI with RHV

Contributors

Mellanox SN2010 switches are used in this deployment procedure to provide 25Gbps connectivity for the data plane of the compute and storage nodes. These steps begin after the switches have been racked, cabled, and put through the initial setup process. To configure the switches to provide data connectivity to the infrastructure, complete the following steps:

Create MLAG Cluster to Provide Fault Tolerance

  1. Run the following commands on each Mellanox SN210 switch for general configuration:

    1. Enter configuration mode.

      Switch-01 enable
Switch-01 configure terminal

    2. Enable the LACP required for the Inter-Peer Link (IPL).

      Switch-01 (config) # lacp

    3. Enable the Link Layer Discovery Protocol (LLDP).

      Switch-01 (config) # lldp

    4. Enable IP routing.

      Switch-01 (config) # ip routing

    5. Enable the MLAG protocol.

      Switch-01 (config) # protocol mlag

    6. Enable global QoS.

      Switch-01 (config) # dcb priority-flow-control enable force

  2. For MLAG to function, the switches must be made peers to each other through an IPL. This should consist of two or more physical links for redundancy. The MTU for the IPL is set for jumbo frames (9216), and all VLANs are enabled by default. Run the following commands on each switch in the domain:

    1. Create port channel 10 for the IPL.

      Switch-01 (config) # interface port-channel 10
Switch-01 (config interface port-channel 10) # description IPL
Switch-01 (config interface port-channel 10) # exit

    2. Add interfaces ETH 1/20 and 1/22 to the port channel.

      Switch-01 (config) # interface ethernet 1/20 channel-group 10 mode active
Switch-01 (config) # interface ethernet 1/20 description ISL-SWB_01
Switch-01 (config) # interface ethernet 1/22 channel-group 10 mode active
Switch-01 (config) # interface ethernet 1/22 description ISL-SWB_02

    3. Create a VLAN outside of the standard range dedicated to IPL traffic.

      Switch-01 (config) # vlan 4000
Switch-01 (config vlan 4000) # name IPL VLAN
Switch-01 (config vlan 4000) # exit

    4. Define the port channel as the IPL.

      Switch-01 (config) # interface port-channel 10 ipl 1
Switch-01 (config) # interface port-channel 10 dcb priority-flow-control mode on force

    5. Set an IP for each IPL member (non-routable; it is not advertised outside of the switch).

      Switch-01 (config) # interface vlan 4000
Switch-01 (config vlan 4000) # ip address 10.0.0.1 255.255.255.0
Switch-01 (config vlan 4000) # ipl 1 peer-address 10.0.0.2
Switch-01 (config vlan 4000) # exit

  3. Create a unique MLAG domain name for the two switches and assign a MLAG virtual IP (VIP). This IP is used for keep-alive heartbeat messages between the two switches. Run these commands on each switch in the domain:

    1. Create the MLAG domain and set the IP address and subnet.

      Switch-01 (config) # mlag-vip MLAG-VIP-DOM ip a.b.c.d /24 force

    2. Create a virtual MAC address for the system MLAG.

      Switch-01 (config) # mlag system-mac AA:BB:CC:DD:EE:FF

    3. Configure the MLAG domain so that it is active globally.

      Switch-01 (config) # no mlag shutdown

The IP used for the MLAG VIP must be in the same subnet as the switch management network (mgmt0). Also, The MAC address used can be any unicast MAC address and must be set to the same value on both switches in the MLAG domain.

Configure Ports to Connect to Storage and Compute Hosts

  1. Create each of the VLANs needed to support the services for NetApp HCI. Run these commands on each switch in the domain:

    1. Create the VLANs.

      Switch-01 (config) # vlan 1172
Switch-01 (config vlan 1172) exit
Switch-01 (config) # vlan 3343
Switch-01 (config vlan 3343) exit
Switch-01 (config) # vlan 3344
Switch-01 (config vlan 3345) exit
Switch-01 (config) # vlan 3345
Switch-01 (config vlan 3346) exit

    2. Create names for each VLAN for easier accounting.

      Switch-01 (config) # vlan 1172 name “MGMT_Network”
Switch-01 (config) # vlan 3343 name “Storage_Network”
Switch-01 (config) # vlan 3345 name “Migration_Network”
Switch-01 (config) # vlan 3346 name “VM_Network”

  2. Create MLAG interfaces and hybrid VLANs on ports identified so that you can distribute connectivity between the switches and tag the appropriate VLANs for the NetApp HCI compute nodes.

    1. Select the ports you want to work with.

      Switch-01 (config) # interface ethernet 1/15

    2. Set the MTU for each port.

      Switch-01 (config interface ethernet 1/15) # mtu 9216 force

    3. Modify spanning- tree settings for each port.

      Switch-01 (config interface ethernet 1/15) # spanning-tree bpdufilter enable
Switch-01 (config interface ethernet 1/15) # spanning-tree port type edge
Switch-01 (config interface ethernet 1/15) # spanning-tree bpduguard enable

    4. Set the switchport mode to hybrid.

      Switch-01 (config interface ethernet 1/15) # switchport mode hybrid
Switch-01 (config interface ethernet 1/15) # exit

    5. Create descriptions for each port being modified.

      Switch-01 (config) # interface ethernet 1/15 description HCI-CMP-01 PortD

    6. Create and configure the MLAG port channels.

      Switch-01 (config) # interface mlag-port-channel 215
Switch-01 (config interface mlag-port-channel 215) # exit
Switch-01 (config) # interface mlag-port-channel 215 no shutdown
Switch-01 (config) # interface mlag-port-channel 215 mtu 9216 force
Switch-01 (config) # interface ethernet 1/15 lacp port-priority 10
Switch-01 (config) # interface ethernet 1/15 lacp rate fast
Switch-01 (config) # interface ethernet 1/15 mlag-channel-group 215 mode active

    7. Tag the appropriate VLANs for the NetApp HCI environment.

      Switch-01 (config) # interface mlag-port-channel 215 switchport hybrid
Switch-01 (config) # interface mlag-port-channel 215 switchport hybrid allowed-vlan add 1172
Switch-01 (config) # interface mlag-port-channel 215 switchport hybrid allowed-vlan add 3343
Switch-01 (config) # interface mlag-port-channel 215 switchport hybrid allowed-vlan add 3345
Switch-01 (config) # interface mlag-port-channel 215 switchport hybrid allowed-vlan add 3346

  3. Create MLAG interfaces and hybrid VLAN ports identified so that you can distribute connectivity between the switches and tag the appropriate VLANs for the NetApp HCI storage nodes.

    1. Select the ports that you want to work with.

      Switch-01 (config) # interface ethernet 1/3

    2. Set the MTU for each port.

      Switch-01 (config interface ethernet 1/3) # mtu 9216 force

    3. Modify spanning tree settings for each port.

      Switch-01 (config interface ethernet 1/3) # spanning-tree bpdufilter enable
Switch-01 (config interface ethernet 1/3) # spanning-tree port type edge
Switch-01 (config interface ethernet 1/3) # spanning-tree bpduguard enable

    4. Set the switchport mode to hybrid.

      Switch-01 (config interface ethernet 1/3) # switchport mode hybrid
Switch-01 (config interface ethernet 1/3) # exit

    5. Create descriptions for each port being modified.

      Switch-01 (config) # interface ethernet 1/3 description HCI-STG-01 PortD

    6. Create and configure the MLAG port channels.

      Switch-01 (config) # interface mlag-port-channel 203
Switch-01 (config interface mlag-port-channel 203) # exit
Switch-01 (config) # interface mlag-port-channel 203 no shutdown
Switch-01 (config) # interface mlag-port-channel 203 mtu 9216 force
Switch-01 (config) # interface mlag-port-channel 203 lacp-individual enable force
Switch-01 (config) # interface ethernet 203 lacp port-priority 10
Switch-01 (config) # interface ethernet 203 lacp rate fast
Switch-01 (config) # interface ethernet 1/3 mlag-channel-group 203 mode active

    7. Tag the appropriate VLANs for the storage environment.

      Switch-01 (config) # interface mlag-port-channel 203 switchport mode hybrid
Switch-01 (config) # interface mlag-port-channel 203 switchport hybrid allowed-vlan add 1172
Switch-01 (config) # interface mlag-port-channel 203 switchport hybrid allowed-vlan add 3343
Note The configurations in this section show the configuration for a single port as example. They must also be run for each additional port connected in the solution, as well as on the associated port of the second switch in the MLAG domain. NetApp recommends that the descriptions for each port are updated to reflect the device ports that are being cabled and configured on the other switch.

  1. Create an MLAG interface to provide uplinks to both Mellanox SN2010 switches from the core network.

    Switch-01 (config) # interface mlag port-channel 201
Switch-01 (config interface mlag port-channel) # description Uplink CORE-SWITCH port PORT
Switch-01 (config interface mlag port-channel) # exit

  2. Configure the MLAG members.

    Switch-01 (config) # interface ethernet 1/1 description Uplink to CORE-SWITCH port PORT
Switch-01 (config) # interface ethernet 1/1 speed 10000 force
Switch-01 (config) # interface mlag-port-channel 201 mtu 9216 force
Switch-01 (config) # interface ethernet 1/1 mlag-channel-group 201 mode active

  3. Set the switchport mode to hybrid and allow all VLANs from the core uplink switches.

    Switch-01 (config) # interface mlag-port-channel switchport mode hybrid
Switch-01 (config) # interface mlag-port-channel switchport hybrid allowed-vlan all

  4. Verify that the MLAG interface is up.

    Switch-01 (config) # interface mlag-port-channel 201 no shutdown
Switch-01 (config) # exit
Note The configurations in this section must also be run on the second switch in the MLAG domain. NetApp recommends that the descriptions for each port are updated to reflect the device ports that are being cabled and configured on the other switch.