To replace the existing CN1610 cluster switches in a cluster with Broadcom-supported BES-53248 cluster switches, you must perform a specific sequence of tasks.
The examples in this procedure use two nodes, each deploying two 10 GbE cluster interconnect ports: e0a and e0b.
This procedure covers the following scenario:
The following command suppresses automatic case creation for two hours:
cluster1::*> system node autosupport invoke -node * -type all -message MAINT=2h
cluster1::*> network interface show -vserver Cluster -fields auto-revert Logical Vserver Interface Auto-revert --------- ------------- ------------ Cluster node1_clus1 true node1_clus2 true node2_clus1 true node2_clus2 true 4 entries were displayed.
The following example displays how many cluster interconnect interfaces have been configured in each node for each cluster interconnect switch:
cluster1::*> network device-discovery show -protocol cdp Node/ Local Discovered Protocol Port Device (LLDP: ChassisID) Interface Platform ----------- ------ ------------------------- ---------------- ---------------- node2 /cdp e0a CL2 0/2 CN1610 e0b CL1 0/2 CN1610 node1 /cdp e0a CL2 0/1 CN1610 e0b CL1 0/1 CN1610 4 entries were displayed.
cluster1::*> network port show -ipspace Cluster Node: node1 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ----------- -------- ------ e0a Cluster Cluster up 9000 auto/10000 healthy false e0b Cluster Cluster up 9000 auto/10000 healthy false Node: node2 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ----------- -------- ------ e0a Cluster Cluster up 9000 auto/10000 healthy false e0b Cluster Cluster up 9000 auto/10000 healthy false 4 entries were displayed.
cluster1::*> network interface show -vserver Cluster Logical Status Network Current Current Is Vserver Interface Admin/Oper Address/Mask Node Port Home ----------- ---------- ---------- ------------------ ------------- ------- ---- Cluster node1_clus1 up/up 169.254.209.69/16 node1 e0a true node1_clus2 up/up 169.254.49.125/16 node1 e0b true node2_clus1 up/up 169.254.47.194/16 node2 e0a true node2_clus2 up/up 169.254.19.183/16 node2 e0b true 4 entries were displayed.
cluster1::*> cluster ping-cluster -node node2 Host is node2 Getting addresses from network interface table... Cluster node1_clus1 169.254.209.69 node1 e0a Cluster node1_clus2 169.254.49.125 node1 e0b Cluster node2_clus1 169.254.47.194 node2 e0a Cluster node2_clus2 169.254.19.183 node2 e0b Local = 169.254.47.194 169.254.19.183 Remote = 169.254.209.69 169.254.49.125 Cluster Vserver Id = 4294967293 Ping status: .... Basic connectivity succeeds on 4 path(s) Basic connectivity fails on 0 path(s) ................ Detected 9000 byte MTU on 4 path(s): Local 169.254.47.194 to Remote 169.254.209.69 Local 169.254.47.194 to Remote 169.254.49.125 Local 169.254.19.183 to Remote 169.254.209.69 Local 169.254.19.183 to Remote 169.254.49.125 Larger than PMTU communication succeeds on 4 path(s) RPC status: 2 paths up, 0 paths down (tcp check) 2 paths up, 0 paths down (udp check)
(CL1)# configure (CL1)(Config)# interface 0/13-0/16 (CL1)(Interface 0/13-0/16)# shutdown (CL1)(Interface 0/13-0/16)# exit (CL1)(Config)# exit (CL1)#
(cs2) # configure (cs2) (Config)# port-channel name 1/2 temp-isl-cn1610 (cs2) (Config)# interface 0/13-0/16 (cs2) (Interface 0/13-0/16)# no spanning-tree edgeport (cs2) (Interface 0/13-0/16)# addport 1/2 (cs2) (Interface 0/13-0/16)# exit (cs2) (Config)# interface lag 2 (cs2) (Interface lag 2)# mtu 9216 (cs2) (Interface lag 2)# port-channel load-balance 7 (cs2) (Config)# exit (cs2) # show port-channel 1/2 Local Interface................................ 1/2 Channel Name................................... temp-isl-cn1610 Link State..................................... Down Admin Mode..................................... Enabled Type........................................... Static Port-channel Min-links......................... 1 Load Balance Option............................ 7 (Enhanced hashing mode) Mbr Device/ Port Port Ports Timeout Speed Active ------- -------------- --------- ------- 0/13 actor/long 10G Full False partner/long 0/14 actor/long 10G Full False partner/long 0/15 actor/long 10G Full False partner/long 0/16 actor/long 10G Full False partner/long
You must then reconnect the disconnected ports on all nodes to the new BES-53248 switch cs2. Refer to the Hardware Universe for approved cabling options.
You must attach appropriate approved cabling connecting port 0/13 to 0/16 on the new BES-53248 switch cs2, to ports 13 to 16 on the existing CN1610 switch CL1.
(CL1)# configure (CL1)(Config)# interface 0/13-0/16 (CL1)(Interface 0/13-0/16,3/1)# no shutdown (CL1)(Interface 0/13-0/16,3/1)# exit (CL1)(Config)# exit (CL1)#
(CL2)# show port-channel 3/1 Local Interface................................ 3/1 Channel Name................................... ISL-LAG Link State..................................... Up Admin Mode..................................... Enabled Type........................................... Static Load Balance Option............................ 7 (Enhanced hashing mode) Mbr Device/ Port Port Ports Timeout Speed Active -------- -------------- ----------- -------- 0/13 actor/long 10 Gb Full True partner/long 0/14 actor/long 10 Gb Full True partner/long 0/15 actor/long 10 Gb Full True partner/long 0/16 actor/long 10 Gb Full True partner/long
(cs2) # show port-channel 1/2 Local Interface................................ 1/2 Channel Name................................... temp-isl-cn1610 Link State..................................... Up Admin Mode..................................... Enabled Type........................................... Static Port-channel Min-links......................... 1 Load Balance Option............................ 7 (Src/Dest MAC, VLAN, EType, incoming port) Mbr Device/ Port Port Ports Timeout Speed Active ------- ------------- --------- ------- 0/13 actor/long 10G Full True partner/long 0/14 actor/long 10G Full True partner/long 0/15 actor/long 10G Full True partner/long 0/16 actor/long 10G Full True partner/long
cluster1::*> network interface show -vserver Cluster Logical Status Network Current Current Is Vserver Interface Admin/Oper Address/Mask Node Port Home ----------- ------------ ---------- ------------------ ------------- ------- ---- Cluster node1_clus1 up/up 169.254.209.69/16 node1 e0a true node1_clus2 up/up 169.254.49.125/16 node1 e0b true node2_clus1 up/up 169.254.47.194/16 node2 e0a true node2_clus2 up/up 169.254.19.183/16 node2 e0b true 4 entries were displayed.
cluster1::*> network port show -ipspace Cluster Node: node1 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ----------- -------- ------ e0a Cluster Cluster up 9000 auto/10000 healthy false e0b Cluster Cluster up 9000 auto/10000 healthy false Node: node2 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ----------- -------- ------ e0a Cluster Cluster up 9000 auto/10000 healthy false e0b Cluster Cluster up 9000 auto/10000 healthy false 4 entries were displayed.
cluster1::*> cluster ping-cluster -node node2 Host is node2 Getting addresses from network interface table... Cluster node1_clus1 169.254.209.69 node1 e0a Cluster node1_clus2 169.254.49.125 node1 e0b Cluster node2_clus1 169.254.47.194 node2 e0a Cluster node2_clus2 169.254.19.183 node2 eob Local = 169.254.47.194 169.254.19.183 Remote = 169.254.209.69 169.254.49.125 Cluster Vserver Id = 4294967293 Ping status: .... Basic connectivity succeeds on 4 path(s) Basic connectivity fails on 0 path(s) ................ Detected 9000 byte MTU on 4 path(s): Local 169.254.47.194 to Remote 169.254.209.69 Local 169.254.47.194 to Remote 169.254.49.125 Local 169.254.19.183 to Remote 169.254.209.69 Local 169.254.19.183 to Remote 169.254.49.125 Larger than PMTU communication succeeds on 4 path(s) RPC status: 2 paths up, 0 paths down (tcp check) 2 paths up, 0 paths down (udp check)
You must then reconnect the disconnected ports on all nodes to the new BES-53248 switch cs1. Refer to the Hardware Universe for approved cabling options.
(cs2) # configure (cs2) (Config)# deleteport 1/2 all (cs2) (Config)# exit (cs2) # write memory This operation may take a few minutes. Management interfaces will not be available during this time. Are you sure you want to save? (y/n) y Config file 'startup-config' created successfully .
cluster1::*> network port show -ipspace Cluster Node: node1 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ----------- -------- ------ e0a Cluster Cluster up 9000 auto/10000 healthy false e0b Cluster Cluster up 9000 auto/10000 healthy false Node: node2 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ----------- -------- ------ e0a Cluster Cluster up 9000 auto/10000 healthy false e0b Cluster Cluster up 9000 auto/10000 healthy false 4 entries were displayed.
cluster1::*> network interface show -vserver Cluster Logical Status Network Current Current Is Vserver Interface Admin/Oper Address/Mask Node Port Home ----------- ------------ ---------- ------------------ --------- ------- ------ Cluster node1_clus1 up/up 169.254.209.69/16 node1 e0a true node1_clus2 up/up 169.254.49.125/16 node1 e0b true node2_clus1 up/up 169.254.47.194/16 node2 e0a true node2_clus2 up/up 169.254.19.183/16 node2 e0b true 4 entries were displayed.
cluster1::*> cluster ping-cluster -node node2 Host is node2 Getting addresses from network interface table... Cluster node1_clus1 169.254.209.69 node1 e0a Cluster node1_clus2 169.254.49.125 node1 e0b Cluster node2_clus1 169.254.47.194 node2 e0a Cluster node2_clus2 169.254.19.183 node2 e0b Local = 169.254.47.194 169.254.19.183 Remote = 169.254.209.69 169.254.49.125 Cluster Vserver Id = 4294967293 Ping status: .... Basic connectivity succeeds on 4 path(s) Basic connectivity fails on 0 path(s) ................ Detected 9000 byte MTU on 4 path(s): Local 169.254.47.194 to Remote 169.254.209.69 Local 169.254.47.194 to Remote 169.254.49.125 Local 169.254.19.183 to Remote 169.254.209.69 Local 169.254.19.183 to Remote 169.254.49.125 Larger than PMTU communication succeeds on 4 path(s) RPC status: 2 paths up, 0 paths down (tcp check) 2 paths up, 0 paths down (udp check)
cluster1::*> network device-discovery show -protocol cdp Node/ Local Discovered Protocol Port Device (LLDP: ChassisID) Interface Platform ----------- ------ ------------------------- ---------------- ---------------- node1 /cdp e0a cs2 0/1 BES-53248 e0b cs1 0/1 BES-53248 node2 /cdp e0a cs2 0/2 BES-53248 e0b cs1 0/2 BES-53248 4 entries were displayed.
cluster::*> system cluster-switch delete –device CL2 cluster::*> system cluster-switch delete –device CL1
cluster::*> system node autosupport invoke -node * -type all -message MAINT=END
See Configuring the cluster switch log collection feature in the Switch Setup and Configuration Guide for Broadcom-supported BES-53248 switches for the steps required to enable cluster health switch log collection used for collecting switch-related log files.