Skip to main content
Cluster and storage switches

Replace a NVIDIA SN2100 storage switch

Contributors netapp-yvonneo netapp-jolieg

You can replace a defective NVIDIA SN2100 storage switch. This is a nondisruptive procedure.

What you'll need

Before installing the Cumulus software and RCFs on a NVIDIA SN2100 storage switch, ensure that:

  • Your system can support NVIDIA SN2100 storage switches.

  • You have downloaded the applicable RCFs.

The Hardware Universe provides full details of supported ports and their configurations.

The existing network configuration must have the following characteristics:

  • Make sure that all troubleshooting steps have been completed to confirm that your switch needs replacing.

  • Management connectivity must exist on both switches.

    Note Make sure that all troubleshooting steps have been completed to confirm that your switch needs replacing.

The replacement NVIDIA SN2100 switch must have the following characteristics:

  • Management network connectivity is functional.

  • Console access to the replacement switch is in place.

  • The appropriate RCF and Cumulus operating system image is loaded onto the switch.

  • Initial customization of the switch is complete.

Procedure summary

This procedure replaces the second NVIDIA SN2100 storage switch sw2 with the new NVIDIA SN2100 switch nsw2. The two nodes are node1 and node2.

Steps to complete:

  • Confirm the switch to be replaced is sw2.

  • Disconnect the cables from switch sw2.

  • Reconnect the cables to switch nsw2.

  • Verify all device configurations on switch nsw2.

Steps
  1. If AutoSupport is enabled on this cluster, suppress automatic case creation by invoking an AutoSupport message:

    system node autosupport invoke -node * -type all - message MAINT=xh

    x is the duration of the maintenance window in hours.

  2. Change the privilege level to advanced, entering y when prompted to continue:

    set -privilege advanced

  3. Check on the health status of the storage node ports to make sure that there is connection to storage switch S1:

    storage port show -port-type ENET

    Show example
    cluster1::*> storage port show -port-type ENET
                                      Speed                     VLAN
    Node           Port Type  Mode    (Gb/s) State    Status      ID
    -------------- ---- ----- ------- ------ -------- --------- ----
    node1
                   e3a  ENET  storage 100    enabled  online      30
                   e3b  ENET  storage   0    enabled  offline     30
                   e7a  ENET  storage   0    enabled  offline     30
                   e7b  ENET  storage 100    enabled  online      30
    node2
                   e3a  ENET  storage 100    enabled  online      30
                   e3b  ENET  storage   0    enabled  offline     30
                   e7a  ENET  storage   0    enabled  offline     30
                   e7b  ENET  storage 100    enabled  online      30
    cluster1::*>
  4. Verify that storage switch sw1 is available:

    network device-discovery show -protocol lldp

    Show example
    cluster1::*> network device-discovery show -protocol lldp
    Node/       Local  Discovered
    Protocol    Port   Device (LLDP: ChassisID)  Interface         Platform
    ----------- ------ ------------------------- ----------------  ----------------
    node1/lldp
                e0M    sw1 (00:ea:bd:68:6a:e8)   Eth1/46           -
                e0b    sw2 (6c:b2:ae:5f:a5:b2)   Ethernet1/16      -
                e0c    SHFFG1827000286 (d0:39:ea:1c:16:92)
                                                 e0a               -
                e0e    sw3 (6c:b2:ae:5f:a5:ba)   Ethernet1/18      -
                e0f    SHFFG1827000286 (00:a0:98:fd:e4:a9)
                                                 e0b               -
                e0g    sw4 (28:ac:9e:d5:4a:9c)   Ethernet1/11      -
                e0h    sw5 (6c:b2:ae:5f:a5:ca)   Ethernet1/22      -
                e1a    sw6 (00:f6:63:10:be:7c)   Ethernet1/33      -
                e1b    sw7 (00:f6:63:10:be:7d)   Ethernet1/34      -
                e2a    sw8 (b8:ce:f6:91:3d:88)   Ethernet1/35      -
    Press <space> to page down, <return> for next line, or 'q' to quit...
    10 entries were displayed.
  5. Run the net show interface command on the working switch to confirm that you can see both nodes and all shelves:

    net show interface

    Show example
    cumulus@sw1:~$ net show interface
    
    State  Name    Spd   MTU    Mode        LLDP                  Summary
    -----  ------  ----  -----  ----------  --------------------  --------------------
    ...
    ...
    UP     swp1    100G  9216   Trunk/L2   node1 (e3a)             Master: bridge(UP)
    UP     swp2    100G  9216   Trunk/L2   node2 (e3a)             Master: bridge(UP)
    UP     swp3    100G  9216   Trunk/L2   SHFFG1826000112 (e0b)   Master: bridge(UP)
    UP     swp4    100G  9216   Trunk/L2   SHFFG1826000112 (e0b)   Master: bridge(UP)
    UP     swp5    100G  9216   Trunk/L2   SHFFG1826000102 (e0b)   Master: bridge(UP)
    UP     swp6    100G  9216   Trunk/L2   SHFFG1826000102 (e0b)   Master: bridge(UP))
    ...
    ...
  6. Verify the shelf ports in the storage system:

    storage shelf port show -fields remote-device, remote-port

    Show example
    cluster1::*> storage shelf port show -fields remote-device, remote-port
    shelf   id  remote-port   remote-device
    -----   --  -----------   -------------
    3.20    0   swp3          sw1
    3.20    1   -             -
    3.20    2   swp4          sw1
    3.20    3   -             -
    3.30    0   swp5          sw1
    3.20    1   -             -
    3.30    2   swp6          sw1
    3.20    3   -             -
    cluster1::*>
  7. Remove all cables attached to storage switch sw2.

  8. Reconnect all cables to the replacement switch nsw2.

  9. Recheck the health status of the storage node ports:

    storage port show -port-type ENET

    Show example
    cluster1::*> storage port show -port-type ENET
                                        Speed                     VLAN
    Node             Port Type  Mode    (Gb/s) State    Status      ID
    ---------------- ---- ----- ------- ------ -------- --------- ----
    node1
                     e3a  ENET  storage 100    enabled  online      30
                     e3b  ENET  storage   0    enabled  offline     30
                     e7a  ENET  storage   0    enabled  offline     30
                     e7b  ENET  storage 100    enabled  online      30
    node2
                     e3a  ENET  storage 100    enabled  online      30
                     e3b  ENET  storage   0    enabled  offline     30
                     e7a  ENET  storage   0    enabled  offline     30
                     e7b  ENET  storage 100    enabled  online      30
    cluster1::*>
  10. Verify that both switches are available:

    net device-discovery show -protocol lldp

    Show example
    cluster1::*> network device-discovery show -protocol lldp
    Node/       Local  Discovered
    Protocol    Port   Device (LLDP: ChassisID)  Interface         Platform
    ----------- ------ ------------------------- ----------------  ----------------
    node1/lldp
                e0M    sw1 (00:ea:bd:68:6a:e8)   Eth1/46           -
                e0b    sw2 (6c:b2:ae:5f:a5:b2)   Ethernet1/16      -
                e0c    SHFFG1827000286 (d0:39:ea:1c:16:92)
                                                 e0a               -
                e0e    sw3 (6c:b2:ae:5f:a5:ba)   Ethernet1/18      -
                e0f    SHFFG1827000286 (00:a0:98:fd:e4:a9)
                                                 e0b               -
                e0g    sw4 (28:ac:9e:d5:4a:9c)   Ethernet1/11      -
                e0h    sw5 (6c:b2:ae:5f:a5:ca)   Ethernet1/22      -
                e1a    sw6 (00:f6:63:10:be:7c)   Ethernet1/33      -
                e1b    sw7 (00:f6:63:10:be:7d)   Ethernet1/34      -
                e2a    sw8 (b8:ce:f6:91:3d:88)   Ethernet1/35      -
    Press <space> to page down, <return> for next line, or 'q' to quit...
    10 entries were displayed.
  11. Verify the shelf ports in the storage system:

    storage shelf port show -fields remote-device, remote-port

    Show example
    cluster1::*> storage shelf port show -fields remote-device, remote-port
    shelf   id    remote-port     remote-device
    -----   --    -----------     -------------
    3.20    0     swp3            sw1
    3.20    1     swp3            nsw2
    3.20    2     swp4            sw1
    3.20    3     swp4            nsw2
    3.30    0     swp5            sw1
    3.20    1     swp5            nsw2
    3.30    2     swp6            sw1
    3.20    3     swp6            nsw2
    cluster1::*>
  12. Change the privilege level back to admin:

    set -privilege admin

  13. If you suppressed automatic case creation, re-enable it by invoking an AutoSupport message:

    system node autosupport invoke -node * -type all -message MAINT=END