Skip to main content
NetApp HCI Solutions

2. Configure Data Switches

Contributors ac-ntap

Mellanox SN2010 switches provide 25Gbps connectivity for the data plane of the compute and storage nodes. To configure the switches to provide data connectivity to the infrastructure, complete the following steps:

Create MLAG cluster to provide fault tolerance

  1. Run the following commands on each Mellanox SN210 switch for general configuration:

    1. Enter configuration mode.

      Switch-01 enable
Switch-01 configure terminal

    2. Enable the LACP required for the Inter-Peer Link (IPL).

      Switch-01 (config) # lacp

    3. Enable the Link Layer Discovery Protocol (LLDP).

      Switch-01 (config) # lldp

    4. Enable IP routing.

      Switch-01 (config) # ip routing

    5. Enable the MLAG protocol.

      Switch-01 (config) # protocol mlag

    6. Enable global QoS.

      Switch-01 (config) # dcb priority-flow-control enable force

  2. For MLAG to function, the switches must be made peers to each other through an IPL. This should consist of two or more physical links for redundancy. The MTU for the IPL is set for jumbo frames (9216), and all VLANs are enabled by default. Run the following commands on each switch in the domain:

    1. Create port channel 10 for the IPL.

      Switch-01 (config) # interface port-channel 10
Switch-01 (config interface port-channel 10) # description IPL
Switch-01 (config interface port-channel 10) # exit

    2. Add interfaces ETH 1/20 and 1/22 to the port channel.

      Switch-01 (config) # interface ethernet 1/20 channel-group 10 mode active
Switch-01 (config) # interface ethernet 1/20 description ISL-SWB_01
Switch-01 (config) # interface ethernet 1/22 channel-group 10 mode active
Switch-01 (config) # interface ethernet 1/22 description ISL-SWB_02

    3. Create a VLAN outside of the standard range dedicated to IPL traffic.

      Switch-01 (config) # vlan 4000
Switch-01 (config vlan 4000) # name IPL VLAN
Switch-01 (config vlan 4000) # exit

    4. Define the port channel as the IPL.

      Switch-01 (config) # interface port-channel 10 ipl 1
Switch-01 (config) # interface port-channel 10 dcb priority-flow-control mode on force

    5. Set an IP for each IPL member (non-routable; it is not advertised outside of the switch).

      Switch-01 (config) # interface vlan 4000
Switch-01 (config vlan 4000) # ip address 10.0.0.1 255.255.255.0
Switch-01 (config vlan 4000) # ipl 1 peer-address 10.0.0.2
Switch-01 (config vlan 4000) # exit

  3. Create a unique MLAG domain name for the two switches and assign an MLAG virtual IP (VIP). This IP is used for keep-alive heartbeat messages between the two switches. Run these commands on each switch in the domain:

    1. Create the MLAG domain and set the IP address and subnet.

      Switch-01 (config) # mlag-vip MLAG-VIP-DOM ip a.b.c.d /24 force

    2. Create a virtual MAC address for the system MLAG.

      Switch-01 (config) # mlag system-mac AA:BB:CC:DD:EE:FF

    3. Configure the MLAG domain so that it is active globally.

      Switch-01 (config) # no mlag shutdown
Note The IP used for the MLAG VIP must be in the same subnet as the switch management network (mgmt0).
Note The MAC address used can be any unicast MAC address and must be set to the same value on both switches in the MLAG domain.

Configure ports to connect to storage and compute hosts

  1. Create each of the VLANs needed to support the services for NetApp HCI. Run these commands on each switch in the domain:

    1. Create VLANs.

      Switch-01 (config) # vlan 1172
Switch-01 (config vlan 1172) exit
Switch-01 (config) # vlan 3480-3482
Switch-01 (config vlan 3480-3482) exit

    2. Create names for each VLAN for easier accounting.

      Switch-01 (config) # vlan 1172 name “VM_Network”
Switch-01 (config) # vlan 3480 name “MGMT_Network”
Switch-01 (config) # vlan 3481 name “Storage_Network”
Switch-01 (config) # vlan 3482 name “vMotion_Network”
+

  2. Create hybrid VLAN ports on ports ETH1/9-10 so that you can tag the appropriate VLANs for the NetApp HCI compute nodes.

    1. Select the ports you want to work with.

      Switch-01 (config) # interface ethernet 1/9-1/10

    2. Set the MTU for each port.

      Switch-01 (config interface ethernet 1/9-1/10) # mtu 9216 force

    3. Modify spanning-tree settings for each port.

      Switch-01 (config interface ethernet 1/9-1/10) # spanning-tree bpdufilter enable
Switch-01 (config interface ethernet 1/9-1/10) # spanning-tree port type edge
Switch-01 (config interface ethernet 1/9-1/10) # spanning-tree bpduguard enable

    4. Set the switchport mode to hybrid.

      Switch-01 (config interface ethernet 1/9-1/10 ) # switchport mode hybrid
Switch-01 (config interface ethernet 1/9-1/10 ) # exit

    5. Create descriptions for each port being modified.

      Switch-01 (config) # interface ethernet 1/9 description HCI-CMP-01 PortD
Switch-01 (config) # interface ethernet 1/10 description HCI-CMP-02 PortD

    6. Tag the appropriate VLANs for the NetApp HCI environment.

      Switch-01 (config) # interface ethernet 1/9 switchport hybrid allowed-vlan add 1172
Switch-01 (config) # interface ethernet 1/9 switchport hybrid allowed-vlan add 3480-3482
Switch-01 (config) # interface ethernet 1/10 switchport hybrid allowed-vlan add 1172
Switch-01 (config) # interface ethernet 1/10 switchport hybrid allowed-vlan add 3480-3482

  3. Create MLAG interfaces and hybrid VLAN ports on ports ETH1/5-8 so that you can distribute connectivity between the switches and tag the appropriate VLANs for the NetApp HCI storage nodes.

    1. Select the ports that you want to work with.

      Switch-01 (config) # interface ethernet 1/5-1/8

    2. Set the MTU for each port.

      Switch-01 (config interface ethernet 1/5-1/8) # mtu 9216 force

    3. Modify spanning tree settings for each port.

      Switch-01 (config interface ethernet 1/5-1/8) # spanning-tree bpdufilter enable
Switch-01 (config interface ethernet 1/5-1/8) # spanning-tree port type edge
Switch-01 (config interface ethernet 1/5-1/8) # spanning-tree bpduguard enable

    4. Set the switchport mode to hybrid.

      Switch-01 (config interface ethernet 1/5-1/8 ) # switchport mode hybrid
Switch-01 (config interface ethernet 1/5-1/8 ) # exit

    5. Create descriptions for each port being modified.

      Switch-01 (config) # interface ethernet 1/5 description HCI-STG-01 PortD
Switch-01 (config) # interface ethernet 1/6 description HCI-STG-02 PortD
Switch-01 (config) # interface ethernet 1/7 description HCI-STG-03 PortD
Switch-01 (config) # interface ethernet 1/8 description HCI-STG-04 PortD

    6. Create and configure the MLAG port channels.

      Switch-01 (config) # interface mlag-port-channel 115-118
Switch-01 (config interface mlag-port-channel 115-118) # exit
Switch-01 (config) # interface mlag-port-channel 115-118 no shutdown
Switch-01 (config) # interface mlag-port-channel 115-118 mtu 9216 force
Switch-01 (config) # interface mlag-port-channel 115-118 lacp-individual enable force
Switch-01 (config) # interface ethernet 1/5-1/8 lacp port-priority 10
Switch-01 (config) # interface ethernet 1/5-1/8 lacp rate fast
Switch-01 (config) # interface ethernet 1/5 mlag-channel-group 115 mode active
Switch-01 (config) # interface ethernet 1/6 mlag-channel-group 116 mode active
Switch-01 (config) # interface ethernet 1/7 mlag-channel-group 117 mode active
Switch-01 (config) # interface ethernet 1/8 mlag-channel-group 118 mode active

    7. Tag the appropriate VLANs for the storage environment.

      Switch-01 (config) # interface mlag-port-channel 115-118 switchport mode hybrid
Switch-01 (config) # interface mlag-port-channel 115 switchport hybrid allowed-vlan add 1172 Switch-01 (config) # interface mlag-port-channel 116 switchport hybrid allowed-vlan add 1172
Switch-01 (config) # interface mlag-port-channel 117 switchport hybrid allowed-vlan add 1172
Switch-01 (config) # interface mlag-port-channel 118 switchport hybrid allowed-vlan add 1172
Switch-01 (config) # interface mlag-port-channel 115 switchport hybrid allowed-vlan add 3481
Switch-01 (config) # interface mlag-port-channel 116 switchport hybrid allowed-vlan add 3481
Switch-01 (config) # interface mlag-port-channel 117 switchport hybrid allowed-vlan add 3481
Switch-01 (config) # interface mlag-port-channel 118 switchport hybrid allowed-vlan add 3481
Note The configurations in this section must also be run on the second switch in the MLAG domain. NetApp recommends that the descriptions for each port are updated to reflect the device ports that are cabled and configured on the other switch.

  1. Create an MLAG interface to provide uplinks to both Mellanox SN2010 switches from the core network.

    Switch-01 (config) # interface mlag port-channel 101
Switch-01 (config interface mlag port-channel) # description Uplink CORE-SWITCH port PORT
Switch-01 (config interface mlag port-channel) # exit

  2. Configure the MLAG members.

    Switch-01 (config) # interface ethernet 1/18 description Uplink to CORE-SWITCH port PORT
Switch-01 (config) # interface ethernet 1/18 speed 10000 force
Switch-01 (config) # interface mlag-port-channel 101 mtu 9216 force
Switch-01 (config) # interface ethernet 1/18 mlag-channel-group 101 mode active

  3. Set the switchport mode to hybrid and allow all VLANs from the core uplink switches.

    Switch-01 (config) # interface mlag-port-channel switchport mode hybrid
Switch-01 (config) # interface mlag-port-channel switchport hybrid allowed-vlan all

  4. Verify that the MLAG interface is up.

    Switch-01 (config) # interface mlag-port-channel 101 no shutdown
Switch-01 (config) # exit

Next: Deploy NetApp HCI with the NetApp Deployment Engine