Replace CN1610 switch CL2 with 3232C switch C2
Follow these steps to replace your CN1610 (CL2) switch with a Nexus 3232C (C2) switch.
-
Shut down the cluster interconnect ports that are physically connected to switch CL2:
network port modify -node node-name -port port-name -up-admin false
Show example
The following example shows the four cluster interconnect ports being shut down for node n1 and node n2:
cluster::*> network port modify -node n1 -port e0b -up-admin false cluster::*> network port modify -node n1 -port e0c -up-admin false cluster::*> network port modify -node n2 -port e0b -up-admin false cluster::*> network port modify -node n2 -port e0c -up-admin false
-
Verify the connectivity of the remote cluster interfaces:
You can use the network interface check cluster-connectivity
command to start an accessibility check for cluster connectivity and then display the details:
network interface check cluster-connectivity start
and network interface check cluster-connectivity show
cluster1::*> network interface check cluster-connectivity start
NOTE: Wait for a number of seconds before running the show
command to display the details.
cluster1::*> network interface check cluster-connectivity show Source Destination Packet Node Date LIF LIF Loss ------ -------------------------- ---------------- ---------------- ----------- n1 3/5/2022 19:21:18 -06:00 n1_clus2 n2-clus1 none 3/5/2022 19:21:20 -06:00 n1_clus2 n2_clus2 none n2 3/5/2022 19:21:18 -06:00 n2_clus2 n1_clus1 none 3/5/2022 19:21:20 -06:00 n2_clus2 n1_clus2 none
For all ONTAP releases, you can also use the cluster ping-cluster -node <name>
command to check the connectivity:
cluster ping-cluster -node <name>
cluster1::*> cluster ping-cluster -node local Host is n1 Getting addresses from network interface table... Cluster n1_clus1 n1 e0a 10.10.0.1 Cluster n1_clus2 n1 e0b 10.10.0.2 Cluster n1_clus3 n1 e0c 10.10.0.3 Cluster n1_clus4 n1 e0d 10.10.0.4 Cluster n2_clus1 n2 e0a 10.10.0.5 Cluster n2_clus2 n2 e0b 10.10.0.6 Cluster n2_clus3 n2 e0c 10.10.0.7 Cluster n2_clus4 n2 e0d 10.10.0.8 Local = 10.10.0.1 10.10.0.2 10.10.0.3 10.10.0.4 Remote = 10.10.0.5 10.10.0.6 10.10.0.7 10.10.0.8 Cluster Vserver Id = 4294967293 Ping status: .... Basic connectivity succeeds on 16 path(s) Basic connectivity fails on 0 path(s) ................ Detected 9000 byte MTU on 16 path(s): Local 10.10.0.1 to Remote 10.10.0.5 Local 10.10.0.1 to Remote 10.10.0.6 Local 10.10.0.1 to Remote 10.10.0.7 Local 10.10.0.1 to Remote 10.10.0.8 Local 10.10.0.2 to Remote 10.10.0.5 Local 10.10.0.2 to Remote 10.10.0.6 Local 10.10.0.2 to Remote 10.10.0.7 Local 10.10.0.2 to Remote 10.10.0.8 Local 10.10.0.3 to Remote 10.10.0.5 Local 10.10.0.3 to Remote 10.10.0.6 Local 10.10.0.3 to Remote 10.10.0.7 Local 10.10.0.3 to Remote 10.10.0.8 Local 10.10.0.4 to Remote 10.10.0.5 Local 10.10.0.4 to Remote 10.10.0.6 Local 10.10.0.4 to Remote 10.10.0.7 Local 10.10.0.4 to Remote 10.10.0.8 Larger than PMTU communication succeeds on 16 path(s) RPC status: 4 paths up, 0 paths down (tcp check) 4 paths up, 0 paths down (udp check)
-
Shut down the ISL ports 13 through 16 on the active CN1610 switch CL1 using the appropriate command.
For more information on Cisco commands, see the guides listed in the Cisco Nexus 3000 Series NX-OS Command References.
Show example
The following example shows ISL ports 13 through 16 being shut down on the CN1610 switch CL1:
(CL1)# configure (CL1)(Config)# interface 0/13-0/16 (CL1)(Interface 0/13-0/16)# shutdown (CL1)(Interface 0/13-0/16)# exit (CL1)(Config)# exit (CL1)#
-
Build a temporary ISL between CL1 and C2:
For more information on Cisco commands, see the guides listed in the Cisco Nexus 3000 Series NX-OS Command References.
Show example
The following example shows a temporary ISL being built between CL1 (ports 13-16) and C2 (ports e1/24/1-4) using the Cisco
switchport mode trunk
command:C2# configure C2(config)# interface port-channel 2 C2(config-if)# switchport mode trunk C2(config-if)# spanning-tree port type network C2(config-if)# mtu 9216 C2(config-if)# interface breakout module 1 port 24 map 10g-4x C2(config)# interface e1/24/1-4 C2(config-if-range)# switchport mode trunk C2(config-if-range)# mtu 9216 C2(config-if-range)# channel-group 2 mode active C2(config-if-range)# exit C2(config-if)# exit
-
Remove the cables that are attached to the CN1610 switch CL2 on all the nodes.
Using supported cabling, you must reconnect the disconnected ports on all the nodes to the Nexus 3232C switch C2.
-
Remove four ISL cables from ports 13 to 16 on the CN1610 switch CL1.
You must attach the appropriate Cisco QSFP28 to SFP+ breakout cables connecting port 1/24 on the new Cisco 3232C switch C2 to ports 13 to 16 on the existing CN1610 switch CL1.
When reconnecting any cables to the new Cisco 3232C switch, the cables used must be either optical fiber or Cisco twinax cables.
-
Make the ISL dynamic by configuring the ISL interface 3/1 on the active CN1610 switch to disable the static mode.
This configuration matches with the ISL configuration on the 3232C switch C2 when the ISLs are brought up on both switches.
For more information on Cisco commands, see the guides listed in the Cisco Nexus 3000 Series NX-OS Command References.
Show example
The following example shows the ISL interface 3/1 being configured to make the ISL dynamic:
(CL1)# configure (CL1)(Config)# interface 3/1 (CL1)(Interface 3/1)# no port-channel static (CL1)(Interface 3/1)# exit (CL1)(Config)# exit (CL1)#
-
Bring up ISLs 13 through 16 on the active CN1610 switch CL1.
For more information on Cisco commands, see the guides listed in the Cisco Nexus 3000 Series NX-OS Command References.
Show example
The following example shows ISL ports 13 through 16 being brought up on the port-channel interface 3/1:
(CL1)# configure (CL1)(Config)# interface 0/13-0/16,3/1 (CL1)(Interface 0/13-0/16,3/1)# no shutdown (CL1)(Interface 0/13-0/16,3/1)# exit (CL1)(Config)# exit (CL1)#
-
Verify that the ISLs are
up
on the CN1610 switch CL1.The "Link State" should be
Up
, "Type" should beDynamic
, and the "Port Active" column should beTrue
for ports 0/13 to 0/16.Show example
The following example shows the ISLs being verified as
up
on the CN1610 switch CL1:(CL1)# show port-channel 3/1 Local Interface................................ 3/1 Channel Name................................... ISL-LAG Link State..................................... Up Admin Mode..................................... Enabled Type........................................... Dynamic Load Balance Option............................ 7 (Enhanced hashing mode) Mbr Device/ Port Port Ports Timeout Speed Active ------ ------------- ---------- ------- 0/13 actor/long 10 Gb Full True partner/long 0/14 actor/long 10 Gb Full True partner/long 0/15 actor/long 10 Gb Full True partner/long 0/16 actor/long 10 Gb Full True partner/long
-
Verify that the ISLs are
up
on the 3232C switch C2:show port-channel summary
For more information on Cisco commands, see the guides listed in the Cisco Nexus 3000 Series NX-OS Command References.
Ports Eth1/24/1 through Eth1/24/4 should indicate
(P)
, meaning that all four ISL ports are up in the port channel. Eth1/31 and Eth1/32 should indicate(D)
as they are not connected.Show example
The following example shows the ISLs being verified as
up
on the 3232C switch C2:C2# show port-channel summary Flags: D - Down P - Up in port-channel (members) I - Individual H - Hot-standby (LACP only) s - Suspended r - Module-removed S - Switched R - Routed U - Up (port-channel) M - Not in use. Min-links not met ------------------------------------------------------------------------------ Group Port- Type Protocol Member Ports Channel ------------------------------------------------------------------------------ 1 Po1(SU) Eth LACP Eth1/31(D) Eth1/32(D) 2 Po2(SU) Eth LACP Eth1/24/1(P) Eth1/24/2(P) Eth1/24/3(P) Eth1/24/4(P)
-
Bring up all of the cluster interconnect ports that are connected to the 3232C switch C2 on all of the nodes:
network port modify -node node-name -port port-name -up-admin true
Show example
The following example shows how to bring up the cluster interconnect ports connected to the 3232C switch C2:
cluster::*> network port modify -node n1 -port e0b -up-admin true cluster::*> network port modify -node n1 -port e0c -up-admin true cluster::*> network port modify -node n2 -port e0b -up-admin true cluster::*> network port modify -node n2 -port e0c -up-admin true
-
Revert all of the migrated cluster interconnect LIFs that are connected to C2 on all of the nodes:
network interface revert -vserver cluster -lif lif-name
Show example
cluster::*> network interface revert -vserver cluster -lif n1_clus2 cluster::*> network interface revert -vserver cluster -lif n1_clus3 cluster::*> network interface revert -vserver cluster -lif n2_clus2 cluster::*> network interface revert -vserver cluster -lif n2_clus3
-
Verify that all of the cluster interconnect ports are reverted to their home ports:
network interface show -role cluster
Show example
The following example shows that the LIFs on clus2 are reverted to their home ports; the LIFs are successfully reverted if the ports in the "Current Port" column have a status of
true
in the "Is Home" column. If the "Is Home" value isfalse
, then the LIF is not reverted.cluster::*> network interface show -role cluster (network interface show) Logical Status Network Current Current Is Vserver Interface Admin/Oper Address/Mask Node Port Home -------- ---------- ----------- -------------- -------- -------- ----- Cluster n1_clus1 up/up 10.10.0.1/24 n1 e0a true n1_clus2 up/up 10.10.0.2/24 n1 e0b true n1_clus3 up/up 10.10.0.3/24 n1 e0c true n1_clus4 up/up 10.10.0.4/24 n1 e0d true n2_clus1 up/up 10.10.0.5/24 n2 e0a true n2_clus2 up/up 10.10.0.6/24 n2 e0b true n2_clus3 up/up 10.10.0.7/24 n2 e0c true n2_clus4 up/up 10.10.0.8/24 n2 e0d true 8 entries were displayed.
-
Verify that all of the cluster ports are connected:
network port show -role cluster
Show example
The following example shows the output verifying all of the cluster interconnects are
up
:cluster::*> network port show -role cluster (network port show) Node: n1 Broadcast Speed (Mbps) Health Ignore Port IPspace Domain Link MTU Admin/Open Status Health Status ----- --------- ----------- ----- ----- ------------ -------- ------------- e0a cluster cluster up 9000 auto/10000 - e0b cluster cluster up 9000 auto/10000 - e0c cluster cluster up 9000 auto/10000 - - e0d cluster cluster up 9000 auto/10000 - - Node: n2 Broadcast Speed (Mbps) Health Ignore Port IPspace Domain Link MTU Admin/Open Status Health Status ----- --------- ----------- ----- ----- ------------ -------- ------------- e0a cluster cluster up 9000 auto/10000 - e0b cluster cluster up 9000 auto/10000 - e0c cluster cluster up 9000 auto/10000 - e0d cluster cluster up 9000 auto/10000 - 8 entries were displayed.
-
Verify the connectivity of the remote cluster interfaces:
You can use the network interface check cluster-connectivity
command to start an accessibility check for cluster connectivity and then display the details:
network interface check cluster-connectivity start
and network interface check cluster-connectivity show
cluster1::*> network interface check cluster-connectivity start
NOTE: Wait for a number of seconds before running the show
command to display the details.
cluster1::*> network interface check cluster-connectivity show Source Destination Packet Node Date LIF LIF Loss ------ -------------------------- ---------------- ---------------- ----------- n1 3/5/2022 19:21:18 -06:00 n1_clus2 n2-clus1 none 3/5/2022 19:21:20 -06:00 n1_clus2 n2_clus2 none n2 3/5/2022 19:21:18 -06:00 n2_clus2 n1_clus1 none 3/5/2022 19:21:20 -06:00 n2_clus2 n1_clus2 none
For all ONTAP releases, you can also use the cluster ping-cluster -node <name>
command to check the connectivity:
cluster ping-cluster -node <name>
cluster1::*> cluster ping-cluster -node local Host is n1 Getting addresses from network interface table... Cluster n1_clus1 n1 e0a 10.10.0.1 Cluster n1_clus2 n1 e0b 10.10.0.2 Cluster n1_clus3 n1 e0c 10.10.0.3 Cluster n1_clus4 n1 e0d 10.10.0.4 Cluster n2_clus1 n2 e0a 10.10.0.5 Cluster n2_clus2 n2 e0b 10.10.0.6 Cluster n2_clus3 n2 e0c 10.10.0.7 Cluster n2_clus4 n2 e0d 10.10.0.8 Local = 10.10.0.1 10.10.0.2 10.10.0.3 10.10.0.4 Remote = 10.10.0.5 10.10.0.6 10.10.0.7 10.10.0.8 Cluster Vserver Id = 4294967293 Ping status: .... Basic connectivity succeeds on 16 path(s) Basic connectivity fails on 0 path(s) ................ Detected 9000 byte MTU on 16 path(s): Local 10.10.0.1 to Remote 10.10.0.5 Local 10.10.0.1 to Remote 10.10.0.6 Local 10.10.0.1 to Remote 10.10.0.7 Local 10.10.0.1 to Remote 10.10.0.8 Local 10.10.0.2 to Remote 10.10.0.5 Local 10.10.0.2 to Remote 10.10.0.6 Local 10.10.0.2 to Remote 10.10.0.7 Local 10.10.0.2 to Remote 10.10.0.8 Local 10.10.0.3 to Remote 10.10.0.5 Local 10.10.0.3 to Remote 10.10.0.6 Local 10.10.0.3 to Remote 10.10.0.7 Local 10.10.0.3 to Remote 10.10.0.8 Local 10.10.0.4 to Remote 10.10.0.5 Local 10.10.0.4 to Remote 10.10.0.6 Local 10.10.0.4 to Remote 10.10.0.7 Local 10.10.0.4 to Remote 10.10.0.8 Larger than PMTU communication succeeds on 16 path(s) RPC status: 4 paths up, 0 paths down (tcp check) 4 paths up, 0 paths down (udp check)
-
Migrate the LIFs that are associated with the first CN1610 switch CL1:
network interface migrate -vserver cluster -lif lif-name -source-node node-name
Show example
You must migrate each cluster LIF individually to the appropriate cluster ports hosted on cluster switch C2 as shown in the following example:
cluster::*> network interface migrate -vserver cluster -lif n1_clus1 -source-node n1 -destination-node n1 -destination-port e0b cluster::*> network interface migrate -vserver cluster -lif n1_clus4 -source-node n1 -destination-node n1 -destination-port e0c cluster::*> network interface migrate -vserver cluster -lif n2_clus1 -source-node n2 -destination-node n2 -destination-port e0b cluster::*> network interface migrate -vserver cluster -lif n2_clus4 -source-node n2 -destination-node n2 -destination-port e0c