Replacing a Broadcom-supported BES-53248 cluster switch

Replacing a defective Broadcom-supported BES-53248 cluster switch in a cluster network is a nondisruptive procedure (NDU).

Before you begin

The following conditions must exist before performing the switch replacement in the current environment and on the replacement switch.

About this task

You must execute the command for migrating a cluster LIF from the node where the cluster LIF is hosted.

The examples in this procedure use the following switch and node nomenclature:

Note: The following procedure is based on the following cluster network topology:
cluster1::> network port show -ipspace Cluster

Node: node1
                                                                       Ignore
                                                  Speed(Mbps) Health   Health
Port      IPspace      Broadcast Domain Link MTU  Admin/Oper  Status   Status
--------- ------------ ---------------- ---- ---- ----------- -------- ------
e0a       Cluster      Cluster          up   9000  auto/10000 healthy  false
e0b       Cluster      Cluster          up   9000  auto/10000 healthy  false


Node: node2
                                                                       Ignore
                                                  Speed(Mbps) Health   Health
Port      IPspace      Broadcast Domain Link MTU  Admin/Oper  Status   Status
--------- ------------ ---------------- ---- ---- ----------- -------- ------
e0a       Cluster      Cluster          up   9000  auto/10000 healthy  false
e0b       Cluster      Cluster          up   9000  auto/10000 healthy  false

4 entries were displayed.

 
cluster1::> network interface show -vserver Cluster
            Logical    Status     Network            Current       Current Is
Vserver     Interface  Admin/Oper Address/Mask       Node          Port    Home
----------- ---------- ---------- ------------------ ------------- ------- ----
Cluster
            node1_clus1  up/up    169.254.209.69/16  node1         e0a     true
            node1_clus2  up/up    169.254.49.125/16  node1         e0b     true
            node2_clus1  up/up    169.254.47.194/16  node2         e0a     true
            node2_clus2  up/up    169.254.19.183/16  node2         e0b     true

4 entries were displayed.



cluster1::> network device-discovery show -protocol cdp
Node/       Local  Discovered
Protocol    Port   Device (LLDP: ChassisID)  Interface         Platform
----------- ------ ------------------------- ----------------  ----------------
node2      /cdp
            e0a    cs1                       0/2               BES-53248
            e0b    cs2                       0/2               BES-53248
node1      /cdp
            e0a    cs1                       0/1               BES-53248
            e0b    cs2                       0/1               BES-53248

4 entries were displayed.


(cs1) # show isdp neighbors

Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge,
                  S - Switch, H - Host, I - IGMP, r - Repeater

Device ID                Intf      Holdtime  Capability Platform         Port ID
------------------------ --------- --------- ---------- ---------------- ---------
node1                    0/1       175       H          FAS2750          e0a
node2                    0/2       152       H          FAS2750          e0a
cs2                      0/55      179       R          BES-53248        0/55
cs2                      0/56      179       R          BES-53248        0/56


(cs2) # show isdp neighbors 

Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge,
                  S - Switch, H - Host, I - IGMP, r - Repeater

Device ID                Intf      Holdtime  Capability Platform         Port ID
------------------------ --------- --------- ---------- ---------------- ---------
node1                    0/1       129       H          FAS2750          e0b
node2                    0/2       165       H          FAS2750          e0b
cs1                      0/55      179       R          BES-53248        0/55
cs1                      0/56      179       R          BES-53248        0/56

Steps

  1. Optional: Install the appropriate RCF and image on the switch, newcs2, and make any necessary site preparations.
    If necessary, verify, download, and install the appropriate versions of the RCF and EFOS software for the new switch. If you have verified that the new switch is correctly set up and does not need updates to the RCF and EFOS software, continue to step 2.
    1. Go to the NetApp Cluster and Management Network Switches Reference Configuration File Description Page on the NetApp Support Site.
    2. Click the link for the Cluster Network and Management Network Compatibility Matrix, and then note the required switch software version.
    3. Click your browser's back arrow to return to the Description page, click CONTINUE, accept the license agreement, and then go to the Download page.
    4. Follow the steps on the Download page to download the correct RCF and EFOS files for the version of ONTAP software you are installing.
  2. On the new switch, log in as admin and shut down all of the ports that will be connected to the node cluster interfaces (ports 1 to 16).
    If the switch that you are replacing is not functional and is powered down, go to Step 4. The LIFs on the cluster nodes should have already failed over to the other cluster port for each node.
    Note: No password is required to enter enable mode.
    Example
    User:admin
    Password:
    (newcs2) >enable
    (newcs2) #config
    (newcs2)(config)#interface 0/1-0/16
    (newcs2)(interface 0/1-0/16)#shutdown
  3. Verify that all cluster LIFs have auto-revert enabled:network interface show -vserver Cluster -fields auto-revert
    Example
    cluster1::> network interface show -vserver Cluster -fields auto-revert
    
    										Logical
    Vserver   Interface    Auto-revert
    --------- ------------ ------------
    Cluster   node1_clus1  true
    Cluster   node1_clus2  true
    Cluster   node2_clus1  true
    Cluster			node2_clus2		true
    
    4 entries were displayed.
    
  4. Shut down the ISL ports 0/55 and 0/56 on the BES-53248 switch cs1:
    Example
    (cs1) #config
    (cs1)(config)#interface 0/55-0/56
    (cs1)(interface 0/55-0/56)#shutdown
  5. Remove all cables from the BES-53248 csc2 switch, and then connect them to the same ports on the BES-53248 newcs2 switch.
  6. Bring up the ISLs ports 0/55 and 0/56 between the cs1 and newcs2 switches, and then verify the port channel operation status.
    The Link State for port-channel 1/1 should be up and all member ports should be True under the Port Active heading.
    Example

    This example enables ISL ports 0/55 and 0/56 and displays the Link State for port-channel 1/1 on switch cs1:

    (cs1) # config
    (cs1)(config)# interface 0/55-0/56
    (cs1)(interface 0/55-0/56)# no shutdown
    (cs1) # show port-channel 1/1
    
    
    Local Interface................................ 1/1
    Channel Name................................... Cluster-ISL
    Link State..................................... Up
    Admin Mode..................................... Enabled
    Type........................................... Dynamic
    Port-channel Min-links......................... 1
    Load Balance Option............................ 7
    (Enhanced hashing mode)
    
    Mbr    Device/       Port       Port
    Ports  Timeout       Speed      Active
    ------ ------------- ---------- -------
    0/55   actor/long    100G Full  True   
           partner/long  
    0/56   actor/long    100G Full  True   
           partner/long  
    
  7. Verify that port e0b is up: network port show -ipspace Cluster
    Example

    The output should be similar to the following:

    cluster1::> network port show -ipspace Cluster
       
    Node: node1
                                                                            Ignore
                                                       Speed(Mbps) Health   Health
    Port      IPspace      Broadcast Domain Link MTU   Admin/Oper  Status   Status
    --------- ------------ ---------------- ---- ----- ----------- -------- -------
    e0a       Cluster      Cluster          up   9000  auto/10000  healthy  false
    e0b       Cluster      Cluster          up   9000  auto/10000  healthy  false
    
    Node: node2
                                                                            Ignore
                                                       Speed(Mbps) Health   Health
    Port      IPspace      Broadcast Domain Link MTU   Admin/Oper  Status   Status
    --------- ------------ ---------------- ---- ----- ----------- -------- -------
    e0a       Cluster      Cluster          up   9000  auto/10000  healthy  false
    e0b       Cluster      Cluster          up   9000  auto/auto   -        false
    
    4 entries were displayed.
  8. On the same node as you used in the previous step, wait for the cluster LIF node1_clus2 on node1 to auto-revert.
    In this example, LIF node1_clus2 on node1 is successfully reverted if Is Home is true and the port is e0b.
    Example

    The following command displays information about the LIFs on both nodes. Bringing up the first node is successful if Is Home is true for both cluster interfaces and they show the correct port assignments, in this example e0a and e0b on node1.

    cluster::> network interface show -vserver Cluster
    
                Logical      Status     Network            Current    Current Is
    Vserver     Interface    Admin/Oper Address/Mask       Node       Port    Home
    ----------- ------------ ---------- ------------------ ---------- ------- -----
    Cluster
                node1_clus1  up/up      169.254.209.69/16  node1      e0a     true
                node1_clus2  up/up      169.254.49.125/16  node1      e0b     true
                node2_clus1  up/up      169.254.47.194/16  node2      e0a     true
                node2_clus2  up/up      169.254.19.183/16  node2      e0a     false
    
    4 entries were displayed.
    
  9. If bringing up the first node is successful, repeat Steps 10 through 13 to bring up the cluster port and revert the cluster interface on the other nodes.
  10. Display information about the nodes in a cluster: cluster show
    Example

    This example shows that the node health for node1 and node2 in this cluster is true:

    cluster1::> cluster show
    Node                   Health Eligibility    Epsilon
    --------------------   ------- ------------  --------
    node1                   true    true           true
    node2                   true    true           true
    
  11. Confirm the following cluster network configuration: network port show
    Example
    cluster1::> network port show -ipspace Cluster
    Node: node1
    																																																																			 	  Ignore
                                           Speed(Mbps)            Health   Health
    Port      IPspace     Broadcast Domain Link MTU   Admin/Oper  Status   Status
    --------- ----------- ---------------- ---- ----- ----------- -------- ------
    e0a       Cluster     Cluster          up   9000  auto/10000  healthy  false
    e0b       Cluster     Cluster          up   9000  auto/10000  healthy  false
    
    Node: node2
                                                                           Ignore
                                            Speed(Mbps)           Health   Health
    Port      IPspace      Broadcast Domain Link MTU  Admin/Oper  Status   Status
    --------- ------------ ---------------- ---- ---- ----------- -------- ------
    e0a       Cluster      Cluster          up   9000 auto/10000  healthy  false
    e0b       Cluster      Cluster          up   9000 auto/10000  healthy  false
    
    4 entries were displayed.
    
    
    cluster1::> network interface show -vserver Cluster
    
                Logical    Status     Network            Current       Current Is
    Vserver     Interface  Admin/Oper Address/Mask       Node          Port    Home
    ----------- ---------- ---------- ------------------ ------------- ------- ----
    Cluster
                node1_clus1  up/up    169.254.209.69/16  node1         e0a     true
                node1_clus2  up/up    169.254.49.125/16  node1         e0b     true
                node2_clus1  up/up    169.254.47.194/16  node2         e0a     true
                node2_clus2  up/up    169.254.19.183/16  node2         e0b     true
    4 entries were displayed.
    
    
    
    cs1# show cdp neighbors
    
    Capability Codes: R - Router, T - Trans-Bridge, B - Source-Route-Bridge
                      S - Switch, H - Host, I - IGMP, r - Repeater,
                      V - VoIP-Phone, D - Remotely-Managed-Device,
                      s - Supports-STP-Dispute
    
    Device-ID            Local Intrfce  Hldtme Capability  Platform      Port ID     
    node1                Eth1/1         144    H           FAS2980       e0a 
    node2                Eth1/2         145    H           FAS2980       e0a           
    newcs2(FDO296348FU)  Eth1/65        176    R S I s     N9K-C92300YC  Eth1/65       
    newcs2(FDO296348FU)  Eth1/66        176    R S I s     N9K-C92300YC  Eth1/66       
           
    
    Total entries displayed: 4
    
    
    cs2# show cdp neighbors
    
    Capability Codes: R - Router, T - Trans-Bridge, B - Source-Route-Bridge
                      S - Switch, H - Host, I - IGMP, r - Repeater,
                      V - VoIP-Phone, D - Remotely-Managed-Device,
                      s - Supports-STP-Dispute
    
    Device-ID          Local Intrfce  Hldtme Capability  Platform      Port ID      
    node1              Eth1/1         139    H           FAS2980       e0b 
    node2              Eth1/2         124    H           FAS2980       e0b          
    cs1(FDO220329KU)   Eth1/65        178    R S I s     N9K-C92300YC  Eth1/65            
    cs1(FDO220329KU)   Eth1/66        178    R S I s     N9K-C92300YC  Eth1/66              
    
    Total entries displayed: 4