MetroCluster IP switches provided by NetApp are NetApp validated. Beginning with ONTAP 9.7, MetroCluster IP configurations can support switches that are not NetApp validated provided that they are compliant with NetApp specifications.
The requirements show how to configure MetroCluster compliant switches without using reference configuration (RCF) files.
Connecting local cluster connections to a MetroCluster compliant switch is not supported.
The following table provides platform-specific network speeds for MetroCluster compliant switches.
| Platform | Network Speed (Gbps) |
|---|---|
| AFF A800 | 40 or 100 |
| AFF A700 | 40 |
| AFF A320 | 100 |
| AFF A300 | 25 |
| AFF A220 | - |
| FAS9000 | 40 |
| FAS8200 | 25 |
| FAS2750 | - |
The examples provided are valid for Cisco NX31xx and NX32xx switches. If other switches are used, these commands can be used as guidance, but the commands might be different. If a feature shown in the examples is not available on the switch, this means that the switch does not meet the minimum requirements and cannot be used to deploy a MetroCluster configuration. This is true for any switch that is connecting a MetroCluster configuration and for all switches on the path between those switches.
A VLAN in each network must be configured. The example shows how to configure a VLAN in network 10.
Example:
# vlan 10
The load balancing policy should be set so that order is preserved.
Example:
# port-channel load-balance src-dst ip-l4port-vlan
You must configure the access and class maps, which map the RDMA and iSCSI traffic to the appropriate classes.
All TCP traffic to and from the port 65200 is mapped to the storage (iSCSI) class. All TCP traffic to and from the port 10006 is mapped to the RDMA class.
Example:
ip access-list storage 10 permit tcp any eq 65200 any 20 permit tcp any any eq 65200 ip access-list rdma 10 permit tcp any eq 10006 any 20 permit tcp any any eq 10006 class-map type qos match-all storage match access-group name storage class-map type qos match-all rdma match access-group name rdma
You must configure the ingress policy. The ingress policy maps the traffic as classified to the different COS groups. In this example, the RDMA traffic is mapped to COS group 5 and iSCSI traffic is mapped to COS group 4.
Example:
policy-map type qos MetroClusterIP_Ingress class rdma set dscp 40 set cos 5 set qos-group 5 class storage set dscp 32 set cos 4 set qos-group 4
You must configure the egress policy on the switch. The egress policy maps the traffic to the egress queues. In this example, RDMA traffic is mapped to queue 5 and iSCSI traffic is mapped to queue 4.
Example:
policy-map type queuing MetroClusterIP_Egress class type queuing c-out-8q-q7 priority level 1 class type queuing c-out-8q-q6 priority level 2 class type queuing c-out-8q-q5 priority level 3 random-detect threshold burst-optimized ecn class type queuing c-out-8q-q4 priority level 4 random-detect threshold burst-optimized ecn class type queuing c-out-8q-q3 priority level 5 class type queuing c-out-8q-q2 priority level 6 class type queuing c-out-8q-q1 priority level 7 class type queuing c-out-8q-q-default bandwidth remaining percent 100 random-detect threshold burst-optimized ecn
You need to configure a switch that has MetroCluster traffic on an ISL but does not connect to any MetroCluster interfaces. In this case, the traffic is already classified and only needs to be mapped to the appropriate queue. In the following example, all of the COS5 traffic is mapped to the class RDMA, and all of the COS4 traffic is mapped to the class iSCSI. Note that this will affect all of the COS5 and COS4 traffic, not only the MetroCluster traffic. If you only want to map the MetroCluster traffic, then you must use the above class maps to identify the traffic using the access groups.
Example:
class-map type qos match-all rdma match cos 5 class-map type qos match-all storage match cos 4
You can configure a 'trunk' mode port when setting an allowed VLAN.
There are two commands, one to set the allowed VLAN list, and one to add to the existing allowed VLAN list.
You can set the allowed VLANs as shown in the example.
Example:
switchport trunk allowed vlan 10
You can add a VLAN to the allowed list as shown in the example.
Example:
switchport trunk allowed vlan add 10
In the example, port-channel 10 is configured for VLAN 10.
Example:
interface port-channel10 switchport mode trunk switchport trunk allowed vlan 10 mtu 9216 service-policy type queuing output MetroClusterIP_Egress
The ISL ports should be configured as part of a port-channel and be assigned the egress queues as shown in the example.
Example:
interface eth1/15-16 switchport mode trunk switchport trunk allowed vlan 10 no lldp transmit no lldp receive mtu 9216 channel-group 10 mode active service-policy type queuing output MetroClusterIP_Egress no shutdown
You might need to configure the node port in breakout mode. In this example, ports 25 and 26 are configured in 4 x 25 Gbps breakout mode.
Example:
interface breakout module 1 port 25-26 map 25g-4x
You might need to configure the MetroCluster interface port speed. The example shows how to configure the speed to "auto".
Example:
speed auto
The following example shows how to fix the speed at 40 Gbps.
Example:
speed 40000
You might need to configure the interface. In the following example, the interface speed is set to "auto".
The port is in access mode in VLAN 10, MTU is set to 9216 and the MetroCluster ingress policy is assigned.
Example:
interface eth1/9 description MetroCluster-IP Node Port speed auto switchport access vlan 10 spanning-tree port type edge spanning-tree bpduguard enable mtu 9216 flowcontrol receive on flowcontrol send on service-policy type qos input MetroClusterIP_Ingress no shutdown
On 25-Gbps ports, the FEC setting might need to be set to "off" as shown in the example.
Example:
fec off