How to migrate CN1610 cluster switches to BES-53248 cluster switches

To replace the existing CN1610 cluster switches in a cluster with Broadcom-supported BES-53248 cluster switches, you must perform a specific sequence of tasks.

About this task

The examples in this procedure use two nodes, each deploying two 10 GbE cluster interconnect ports: e0a and e0b.

The examples in this procedure use the following switch and node nomenclature:
  • The command outputs might vary depending on different releases of ONTAP software.
  • The CN1610 switches to be replaced are CL1 and CL2.
  • The BES-53248 switches to replace the CN1610 switches are cs1 and cs2.
  • The nodes are node1 and node2.
  • The switch CL1 is replaced by cs1 first, followed with CL2 by cs2.
  • The BES-53248 switches are pre-loaded with the supported versions of Reference Configuration File (RCF) and Ethernet Fabric OS (EFOS) with ISL cables connected on ports 55 and 56.
  • The cluster LIF names are node1_clus1 and node1_clus2 for node1, and node2_clus1 and node2_clus2 for node2.

This procedure covers the following scenario:

Steps

  1. Change the privilege level to advanced, entering y when prompted to continue: set -privilege advanced
    The advanced prompt (*>) appears.
  2. If AutoSupport is enabled on this cluster, suppress automatic case creation by invoking an AutoSupport message: system node autosupport invoke -node * -type all -message MAINT=xh
    where x is the duration of the maintenance window in hours.
    Note: The AutoSupport message notifies technical support of this maintenance task so that automatic case creation is suppressed during the maintenance window.
    Example

    The following command suppresses automatic case creation for two hours:

    cluster1::*> system node autosupport invoke -node * -type all -message MAINT=2h
  3. Verify that auto-revert is enabled on all cluster LIFs: network interface show -vserver Cluster -fields auto-revert
    Example
    cluster1::*> network interface show -vserver Cluster -fields auto-revert
    
              Logical       
    Vserver   Interface     Auto-revert
    --------- ------------- ------------ 
    Cluster
              node1_clus1   true
              node1_clus2   true
              node2_clus1   true
              node2_clus2   true
    
    4 entries were displayed.
    
  4. Display information about the devices in your configuration: network device-discovery show -protocol cdp
    Example

    The following example displays how many cluster interconnect interfaces have been configured in each node for each cluster interconnect switch:

    cluster1::*> network device-discovery show -protocol cdp
    Node/       Local  Discovered
    Protocol    Port   Device (LLDP: ChassisID)  Interface         Platform
    ----------- ------ ------------------------- ----------------  ----------------
    node2      /cdp
                e0a    CL1                       0/2               CN1610
                e0b    CL2                       0/2               CN1610
    node1      /cdp
                e0a    CL1                       0/1               CN1610
                e0b    CL2                       0/1               CN1610
    
    4 entries were displayed.
  5. Determine the administrative or operational status for each cluster interface.
    1. Display the cluster network port attributes: network port show -ipspace Cluster
      Example
      cluster1::*> network port show -ipspace Cluster
      
      Node: node1
                                                                             Ignore
                                                        Speed(Mbps) Health   Health
      Port      IPspace      Broadcast Domain Link MTU  Admin/Oper  Status   Status
      --------- ------------ ---------------- ---- ---- ----------- -------- ------
      e0a       Cluster      Cluster          up   9000  auto/10000 healthy  false
      e0b       Cluster      Cluster          up   9000  auto/10000 healthy  false
      
      Node: node2
                                                                             Ignore
                                                        Speed(Mbps) Health   Health
      Port      IPspace      Broadcast Domain Link MTU  Admin/Oper  Status   Status
      --------- ------------ ---------------- ---- ---- ----------- -------- ------
      e0a       Cluster      Cluster          up   9000  auto/10000 healthy  false
      e0b       Cluster      Cluster          up   9000  auto/10000 healthy  false
      
      4 entries were displayed.
    2. Display information about the logical interfaces: network port show -vserver Cluster
      Example
      cluster1::*> network interface show -vserver Cluster
      
                  Logical    Status     Network            Current       Current Is
      Vserver     Interface  Admin/Oper Address/Mask       Node          Port    Home
      ----------- ---------- ---------- ------------------ ------------- ------- ----
      Cluster
                  node1_clus1  up/up    169.254.209.69/16  node1         e0a     true
                  node1_clus2  up/up    169.254.49.125/16  node1         e0b     true
                  node2_clus1  up/up    169.254.47.194/16  node2         e0a     true
                  node2_clus2  up/up    169.254.19.183/16  node2         e0b     true
      4 entries were displayed.
  6. Verify that the appropriate RCF and image are installed on the new BES-53248 switches as necessary for your requirements, and make any essential site customizations, such as users and passwords, network addresses, and so on.
  7. Ping the remote cluster interfaces: cluster ping-cluster -node node-name
    Example
    The following example shows how to ping the remote cluster interfaces:
    cluster1::*> cluster ping-cluster -node node2
    
    Host is node2
    Getting addresses from network interface table...
    Cluster node1_clus1 169.254.209.69  node1     e0a       
    Cluster node1_clus2 169.254.49.125  node1     e0b       
    Cluster node2_clus1 169.254.47.194  node2     e0a       
    Cluster node2_clus2 169.254.19.183  node2     e0b
          
    Local = 169.254.47.194 169.254.19.183
    Remote = 169.254.209.69 169.254.49.125
    Cluster Vserver Id = 4294967293
    Ping status:
    .... 
    Basic connectivity succeeds on 4 path(s)
    Basic connectivity fails on 0 path(s)
    ................ 
    Detected 9000 byte MTU on 4 path(s):
        Local 169.254.47.194 to Remote 169.254.209.69
        Local 169.254.47.194 to Remote 169.254.49.125
        Local 169.254.19.183 to Remote 169.254.209.69
        Local 169.254.19.183 to Remote 169.254.49.125
    Larger than PMTU communication succeeds on 4 path(s)
    
    RPC status:
    2 paths up, 0 paths down (tcp check)
    2 paths up, 0 paths down (udp check)
  8. Shut down the ISL ports 13 through 16 on the active CN1610 switch CL2: shutdown
    Example
    The following example shows how to shut down ISL ports 13 through 16 on the CN1610 switch CL2:
    (CL2)# configure
    (CL2)(Config)# interface 0/13-0/16
    (CL2)(Interface 0/13-0/16)# shutdown
    (CL2)(Interface 0/13-0/16)# exit
    (CL2)(Config)# exit
    (CL2)#
    
  9. Build a temporary ISL between CN1610 CL2 and new BES-53248 cs1. The ISL will only be defined on cs1 as the existing ISL on CL2 can be reused.
    Example
    The following example builds a temporary ISL on cs1 (ports 13-16) to be connected to the existing ISL on CL2 (ports 13-16):
    (cs1) # configure
    (cs1) (Config)# port-channel name 1/2 temp-isl-cn1610
    (cs1) (Config)# interface 0/13-0/16
    (cs1) (Interface 0/13-0/16)# no spanning-tree edgeport
    (cs1) (Interface 0/13-0/16)# addport 1/2
    (cs1) (Interface 0/13-0/16)# exit
    (cs1) (Config)# interface lag 2
    (cs1) (Interface lag 2)# mtu 9216
    (cs1) (Interface lag 2)# port-channel load-balance 7
    (cs1) (Config)# exit
    
    (cs1) # show port-channel 1/2
    Local Interface................................ 1/2
    Channel Name................................... temp-isl-cn1610
    Link State..................................... Down
    Admin Mode..................................... Enabled
    Type........................................... Static
    Port-channel Min-links......................... 1
    Load Balance Option............................ 7
    (Enhanced hashing mode)
    
    Mbr     Device/        Port      Port
    Ports   Timeout        Speed     Active
    ------- -------------- --------- -------
    0/13    actor/long     10G Full  False        
            partner/long
    0/14    actor/long     10G Full  False 
            partner/long
    0/15    actor/long     10G Full  False
            partner/long
    0/16    actor/long     10G Full  False
            partner/long
    
  10. On all nodes, remove the cables that are attached to the CN1610 switch CL1.

    You must then reconnect the disconnected ports on all nodes to the new BES-53248 switch cs1. Refer to the Hardware Universe for approved cabling options.

  11. Remove four ISL cables from ports 13 to 16 on the CN1610 switch CL1.

    You must attach appropriate approved cabling connecting port 0/13 to 0/16 on the new BES-53248 switch cs1, to ports 13 to 16 on existing CN1610 switch CL2.

  12. Bring up ISLs 13 through 16 on the active CN1610 switch CL2.
    Example
    The following example illustrates the process of bringing up ISL ports 13 through 16 on CL2:
    (CL2)# configure
    (CL2)(Config)# interface 0/13-0/16
    (CL2)(Interface 0/13-0/16,3/1)# no shutdown
    (CL2)(Interface 0/13-0/16,3/1)# exit
    (CL2)(Config)# exit
    (CL2)#
  13. Verify that the ISLs are up on the CN1610 switch CL2: show port-channel
    Example
    The Link State should be Up, Type should be Static, and Port Active should be True for ports 0/13 to 0/16:
    (CL1)# show port-channel 3/1
    Local Interface................................ 3/1
    Channel Name................................... ISL-LAG
    Link State..................................... Up
    Admin Mode..................................... Enabled
    Type........................................... Static
    Load Balance Option............................ 7
    
    
    (Enhanced hashing mode)
    Mbr      Device/        Port        Port
    Ports    Timeout        Speed       Active
    -------- -------------- ----------- --------
    0/13     actor/long     10 Gb Full  True
             partner/long
    0/14     actor/long     10 Gb Full  True
             partner/long
    0/15     actor/long     10 Gb Full  True
             partner/long
    0/16     actor/long     10 Gb Full  True
             partner/long
    
  14. Verify that the ISL ports are up on the BES-53248 switch: show port-channel
    Example
    (cs1) # show port-channel 1/2
    
    Local Interface................................ 1/2
    Channel Name................................... temp-isl-cn1610
    Link State..................................... Up
    Admin Mode..................................... Enabled
    Type........................................... Static
    Port-channel Min-links......................... 1
    Load Balance Option............................ 7
    
    (Src/Dest MAC, VLAN, EType, incoming port)
    
    Mbr     Device/       Port      Port
    Ports   Timeout       Speed     Active
    ------- ------------- --------- -------
    0/13    actor/long    10G Full  True
            partner/long
    0/14    actor/long    10G Full  True
            partner/long
    0/15    actor/long    10G Full  True
            partner/long
    0/16    actor/long    10G Full  True
            partner/long
  15. Verify that all of the cluster interconnect ports are reverted to their home ports: network interface show -vserver Cluster
    Example
    cluster1::*> network interface show -vserver Cluster
                Logical      Status     Network            Current       Current Is
    Vserver     Interface    Admin/Oper Address/Mask       Node          Port    Home
    ----------- ------------ ---------- ------------------ ------------- ------- ----
    Cluster
                node1_clus1  up/up      169.254.209.69/16  node1         e0a     true
                node1_clus2  up/up      169.254.49.125/16  node1         e0b     true
                node2_clus1  up/up      169.254.47.194/16  node2         e0a     true
                node2_clus2  up/up      169.254.19.183/16  node2         e0b     true
    4 entries were displayed.
    
  16. Verify that all of the cluster ports are connected: network port show -ipspace Cluster
    Example
    The following example shows the result of the previous command, verifying that all of the cluster interconnects are up:
    cluster1::*> network port show -ipspace Cluster
    
    Node: node1
                                                                           Ignore
                                                      Speed(Mbps) Health   Health
    Port      IPspace      Broadcast Domain Link MTU  Admin/Oper  Status   Status
    --------- ------------ ---------------- ---- ---- ----------- -------- ------
    e0a       Cluster      Cluster          up   9000  auto/10000 healthy  false
    e0b       Cluster      Cluster          up   9000  auto/10000 healthy  false
    
    Node: node2
                                                                           Ignore
                                                      Speed(Mbps) Health   Health
    Port      IPspace      Broadcast Domain Link MTU  Admin/Oper  Status   Status
    --------- ------------ ---------------- ---- ---- ----------- -------- ------
    e0a       Cluster      Cluster          up   9000  auto/10000 healthy  false
    e0b       Cluster      Cluster          up   9000  auto/10000 healthy  false
    
    4 entries were displayed.
  17. Ping the remote cluster interfaces: cluster ping-cluster -node node-name
    Example
    The following example shows how to ping the remote cluster interfaces:
    cluster1::*> cluster ping-cluster -node node2
    Host is node2
    Getting addresses from network interface table...
    Cluster node1_clus1 169.254.209.69  node1     e0a       
    Cluster node1_clus2 169.254.49.125  node1     e0b       
    Cluster node2_clus1 169.254.47.194  node2     e0a       
    Cluster node2_clus2 169.254.19.183  node2     eob       
    Local = 169.254.47.194 169.254.19.183
    Remote = 169.254.209.69 169.254.49.125
    Cluster Vserver Id = 4294967293
    Ping status:
    .... 
    Basic connectivity succeeds on 4 path(s)
    Basic connectivity fails on 0 path(s)
    ................ 
    Detected 9000 byte MTU on 4 path(s):
        Local 169.254.47.194 to Remote 169.254.209.69
        Local 169.254.47.194 to Remote 169.254.49.125
        Local 169.254.19.183 to Remote 169.254.209.69
        Local 169.254.19.183 to Remote 169.254.49.125
    Larger than PMTU communication succeeds on 4 path(s)
    RPC status:
    2 paths up, 0 paths down (tcp check)
    2 paths up, 0 paths down (udp check)
  18. On all nodes, remove the cables that are attached to the CN1610 switch CL2.

    You must then reconnect the disconnected ports on all nodes to the new BES-53248 switch cs2. Refer to the Hardware Universe for approved cabling options.

  19. Remove four ISL cables from ports 13 to 16 on BES-53248 switch cs1.
  20. Remove the temporary port-channel 2 on cs1.
    Example
    The following example removes port-channel 2 and copies the running-configuration file to the startup-configuration file:
    (cs1) # configure
    (cs1) (Config)# deleteport 1/2 all
    (cs1) (Config)# exit
    (cs1) # write memory
    
    This operation may take a few minutes.
    Management interfaces will not be available during this time.
    
    Are you sure you want to save? (y/n) y
    
    Config file 'startup-config' created successfully .
    
  21. Verify the status of the cluster node port: network port show -ipspace Cluster
    Example
    The following example verifies that all of the cluster interconnect ports on node1 and node2 are up:
    cluster1::*> network port show -ipspace Cluster
    
    Node: node1
                                                                           Ignore
                                                      Speed(Mbps) Health   Health
    Port      IPspace      Broadcast Domain Link MTU  Admin/Oper  Status   Status
    --------- ------------ ---------------- ---- ---- ----------- -------- ------
    e0a       Cluster      Cluster          up   9000  auto/10000 healthy  false
    e0b       Cluster      Cluster          up   9000  auto/10000 healthy  false
    
    Node: node2
                                                                           Ignore
                                                      Speed(Mbps) Health   Health
    Port      IPspace      Broadcast Domain Link MTU  Admin/Oper  Status   Status
    --------- ------------ ---------------- ---- ---- ----------- -------- ------
    e0a       Cluster      Cluster          up   9000  auto/10000 healthy  false
    e0b       Cluster      Cluster          up   9000  auto/10000 healthy  false
    
    4 entries were displayed.
    
  22. Verify that the interface is now home: network interface show -vserver Cluster
    Example
    The following example shows the status of cluster interconnect interfaces are up and Is home for node1 and node2:
    cluster1::*> network interface show -vserver Cluster
                Logical      Status     Network            Current   Current Is
    Vserver     Interface    Admin/Oper Address/Mask       Node      Port    Home
    ----------- ------------ ---------- ------------------ --------- ------- ------
    Cluster
                node1_clus1  up/up      169.254.209.69/16  node1     e0a     true
                node1_clus2  up/up      169.254.49.125/16  node1     e0b     true
                node2_clus1  up/up      169.254.47.194/16  node2     e0a     true
                node2_clus2  up/up      169.254.19.183/16  node2     e0b     true
    
    4 entries were displayed.
  23. Ping the remote cluster interfaces and then perform a remote procedure call server check: cluster ping-cluster -node node-name
    Example
    The following example shows how to ping the remote cluster interfaces:
    cluster1::*> cluster ping-cluster -node node2
    Host is node2
    Getting addresses from network interface table...
    Cluster node1_clus1 169.254.209.69  node1     e0a       
    Cluster node1_clus2 169.254.49.125  node1     e0b       
    Cluster node2_clus1 169.254.47.194  node2     e0a       
    Cluster node2_clus2 169.254.19.183  node2     e0b       
    Local = 169.254.47.194 169.254.19.183
    Remote = 169.254.209.69 169.254.49.125
    Cluster Vserver Id = 4294967293
    Ping status:
    .... 
    Basic connectivity succeeds on 4 path(s)
    Basic connectivity fails on 0 path(s)
    ................ 
    Detected 9000 byte MTU on 4 path(s):
        Local 169.254.47.194 to Remote 169.254.209.69
        Local 169.254.47.194 to Remote 169.254.49.125
        Local 169.254.19.183 to Remote 169.254.209.69
        Local 169.254.19.183 to Remote 169.254.49.125
    Larger than PMTU communication succeeds on 4 path(s)
    RPC status:
    2 paths up, 0 paths down (tcp check)
    2 paths up, 0 paths down (udp check)
  24. Display the information about the devices in your configuration: network device-discovery show -protocol cdp
    Example
    The following examples show node1 and node2 have been migrated from CN1610 CL1 and CL2 to BES-53248 cs1 and cs2:
    cluster1::*> network device-discovery show -protocol cdp
    Node/       Local  Discovered
    Protocol    Port   Device (LLDP: ChassisID)  Interface         Platform
    ----------- ------ ------------------------- ----------------  ----------------
    node1      /cdp
                e0a    cs1                       0/1               BES-53248
                e0b    cs2                       0/1               BES-53248
    node2      /cdp
                e0a    cs1                       0/2               BES-53248
                e0b    cs2                       0/2               BES-53248
    
    4 entries were displayed.
  25. Remove the replaced CN1610 switches if they are not automatically removed: system cluster-switch delete -device device-name
    Example
    The following example shows how to remove the CN1610 switches:
    cluster::*> system cluster-switch delete –device CL1
    cluster::*> system cluster-switch delete –device CL2
  26. If you suppressed automatic case creation, re-enable it by invoking an AutoSupport message: system node autosupport invoke -node * -type all -message MAINT=END
    Example
    cluster::*> system node autosupport invoke -node * -type all -message MAINT=END