Skip to main content
Cluster and storage switches

Prepare to install NX-OS software and RCF

Contributors netapp-yvonneo
PDFs

Before you install the NX-OS software and the Reference Configuration File (RCF), follow this procedure.

About the examples

The examples in this procedure use the following switch and node nomenclature:

  • The names of the two Cisco switches are cs1 and cs2.

  • The node names are node1-01, node1-02, node1-03, and node1-04.

  • The cluster LIF names are:

    • node1-01_clus1 and node1-01_clus2 for node1-01

    • node1-02_clus1 and node1-02_clus2 for node1-02

    • node1-03_clus1 and node1-03_clus2 for node1-03

    • node1-04_clus1 and node1-04_clus2 for node1-04

  • The cluster1::*> prompt indicates the name of the cluster.

About this task

The procedure requires the use of both ONTAP commands and Cisco Nexus 9000 Series Switches commands; ONTAP commands are used unless otherwise indicated.

Steps

  1. If AutoSupport is enabled on this cluster, suppress automatic case creation by invoking an AutoSupport message: system node autosupport invoke -node * -type all -message MAINT=x h

    where x is the duration of the maintenance window in hours.

    Note The AutoSupport message notifies technical support of this maintenance task so that automatic case creation is suppressed during the maintenance window.

  2. Change the privilege level to advanced, entering y when prompted to continue:

    set -privilege advanced

    The advanced prompt (*>) appears.

  3. Check the administrative and operational status of each port.

    1. Cluster ports

      1. Display how many cluster interconnect interfaces are configured in each node for each switch:

        network device-discovery show -protocol cdp
        Show example 
        cluster1::*> network device-discovery show -protocol cdp
Node/       Local  Discovered
Protocol    Port   Device (LLDP: ChassisID) Interface         Platform
----------- ------ ------------------------ ----------------  ----------------
node1-01/cdp
            e10a   cs1(FLMXXXXXXXX)         Ethernet1/16/3    N9K-C9332D-GX2B
            e10b   cs2(FDOXXXXXXXX)         Ethernet1/16/3    N9K-C9332D-GX2B
            e11a   cs1(FLMXXXXXXXX)         Ethernet1/16/4    N9K-C9332D-GX2B
            e11b   cs2(FDOXXXXXXXX)         Ethernet1/16/4    N9K-C9332D-GX2B
            e1a    cs1(FLMXXXXXXXX)         Ethernet1/16/1    N9K-C9332D-GX2B
            e1b    cs2(FDOXXXXXXXX)         Ethernet1/16/1    N9K-C9332D-GX2B
            .
            .
            .
            e7a    cs1(FLMXXXXXXXX)         Ethernet1/16/2    N9K-C9332D-GX2B
            e7b    cs2(FDOXXXXXXXX)         Ethernet1/16/2    N9K-C9332D-GX2B

node1-02/cdp
            e10a   cs1(FLMXXXXXXXX)         Ethernet1/16/3    N9K-C9332D-GX2B
            e10b   cs2(FDOXXXXXXXX)         Ethernet1/16/3    N9K-C9332D-GX2B
            e11a   cs1(FLMXXXXXXXX)         Ethernet1/16/4    N9K-C9332D-GX2B
            e11b   cs2(FDOXXXXXXXX)         Ethernet1/16/4    N9K-C9332D-GX2B
            e1a    cs1(FLMXXXXXXXX)         Ethernet1/16/1    N9K-C9332D-GX2B
            e1b    cs2(FDOXXXXXXXX)         Ethernet1/16/1    N9K-C9332D-GX2B
            .
            .
            .
            e7a    cs1(FLM284504N6)         Ethernet1/16/2    N9K-C9332D-GX2B
            e7b    cs1(FDO2846056X)         Ethernet1/16/2    N9K-C9332D-GX2B
.
.
.

      2. Display the network port attributes:

        network port show -ipspace Cluster
        Show example 
        cluster1::*> network port show -ipspace Cluster

Node: node1-01
                                                                       Ignore
                                                  Speed(Mbps) Health   Health
Port      IPspace      Broadcast Domain Link MTU  Admin/Oper  Status   Status
--------- ------------ ---------------- ---- ---- ----------- -------- ------
e7a       Cluster      Cluster          up   9000  auto/100000 healthy false
e7b       Cluster      Cluster          up   9000  auto/100000 healthy false


Node: node1-02
                                                                       Ignore
                                                  Speed(Mbps) Health   Health
Port      IPspace      Broadcast Domain Link MTU  Admin/Oper  Status   Status
--------- ------------ ---------------- ---- ---- ----------- -------- ------
e7a       Cluster      Cluster          up   9000  auto/100000 healthy false
e7b       Cluster      Cluster          up   9000  auto/100000 healthy false


Node: node1-03
                                                                       Ignore
                                                  Speed(Mbps) Health   Health
Port      IPspace      Broadcast Domain Link MTU  Admin/Oper  Status   Status
--------- ------------ ---------------- ---- ---- ----------- -------- ------
e7a       Cluster      Cluster          up   9000  auto/100000 healthy false
e7b       Cluster      Cluster          up   9000  auto/100000 healthy false


Node: node1-04
                                                                       Ignore
                                                  Speed(Mbps) Health   Health
Port      IPspace      Broadcast Domain Link MTU  Admin/Oper  Status   Status
--------- ------------ ---------------- ---- ---- ----------- -------- ------
e7a       Cluster      Cluster          up   9000  auto/100000 healthy false
e7b       Cluster      Cluster          up   9000  auto/100000 healthy false

8 entries were displayed.

      3. Display information about the cluster LIFs:

        network interface show -vserver Cluster
        Show example 
        cluster1::*> network interface show -vserver Cluster

            Logical       Status     Network            Current   Current Is
Vserver     Interface     Admin/Oper Address/Mask       Node      Port    Home
----------- ------------- ---------- ------------------ --------- ------- ----
Cluster
            node1-01_clus1 up/up     169.254.36.44/16   node1-01  e7a     true
            node1-01_clus2 up/up     169.254.7.5/16     node1-01  e7b     true
            node1-02_clus1 up/up     169.254.197.206/16 node1-02  e7a     true
            node1-02_clus2 up/up     169.254.195.186/16 node1-02  e7b     true
            node1-03_clus1 up/up     169.254.192.49/16  node1-03  e7a     true
            node1-03_clus2 up/up     169.254.182.76/16  node1-03  e7b     true
            node1-04_clus1 up/up     169.254.59.49/16   node1-04  e7a     true
            node1-04_clus2 up/up     169.254.62.244/16  node1-04  e7b     true

8 entries were displayed.

    2. HA ports

      1. Display information about the HA ports:

        ha interconnect status show -node <node-name>

        Show example 
        cluster1::*> ha interconnect status show -node node1-01
  (system ha interconnect status show)

                       Node: node1-01
              Link 0 Status: up
              Link 1 Status: up
           Is Link 0 Active: true
           Is Link 1 Active: true
         IC RDMA Connection: up
                       Slot: 0
             Debug Firmware: no


Interconnect Port 0 :
                  Port Name: e1a-17
                        MTU: 4096
           Link Information: ACTIVE


Interconnect Port 1 :
                  Port Name: e1b-18
                        MTU: 4096
           Link Information: ACTIVE

cluster1::*> ha interconnect status show -node node1-02
  (system ha interconnect status show)

                       Node: node1-02
              Link 0 Status: up
              Link 1 Status: up
           Is Link 0 Active: true
           Is Link 1 Active: true
         IC RDMA Connection: up
                       Slot: 0
             Debug Firmware: no


Interconnect Port 0 :
                  Port Name: e1a-17
                        MTU: 4096
           Link Information: ACTIVE


Interconnect Port 1 :
                  Port Name: e1b-18
                        MTU: 4096
           Link Information: ACTIVE
.
.
.

    3. Storage ports

      1. Display information about the storage ports:

        storage port show -port-type ENET
        Show example 
        cluster1::*> storage port show -port-type ENET

                                      Speed
Node               Port Type  Mode    (Gb/s) State    Status
------------------ ---- ----- ------- ------ -------- -----------
node1-01
                   e10a ENET  -          100 enabled  online
                   e10b ENET  -          100 enabled  online
                   e11a ENET  -          100 enabled  online
                   e11b ENET  -          100 enabled  online
node1-02
                   e10a ENET  -          100 enabled  online
                   e10b ENET  -          100 enabled  online
                   e11a ENET  -          100 enabled  online
                   e11b ENET  -          100 enabled  online
node1-03
                   e10a ENET  -          100 enabled  online
                   e10b ENET  -          100 enabled  online
                   e11a ENET  -          100 enabled  online
node1-04
                   e10a ENET  -          100 enabled  online
                   e10b ENET  -          100 enabled  online
                   e11a ENET  -          100 enabled  online
                   e11b ENET  -          100 enabled  online
16 entries were displayed.

    4. Storage shelf ports

      1. Verify that all the storage shelf ports are up with a healthy status:

        storage shelf port show
        Show example 
        cluster1::*> storage shelf port show

Shelf ID Module State        Internal?
----- -- ------ ------------ ---------
1.1
       0 A      connected    false
       1 A      connected    false
       2 A      connected    false
       3 A      connected    false
       4 A      connected    false
       5 A      connected    false
       6 A      connected    false
       7 A      connected    false
       8 B      connected    false
       9 B      connected    false
      10 B      connected    false
      11 B      connected    false
      12 B      connected    false
      13 B      connected    false
      14 B      connected    false
      15 B      connected    false

16 entries were displayed.

      2. Verify the connection status of all the storage shelf ports:

        storage shelf port show -fields remote-device,remote-port,connector-state
        Show example 
        cluster1::*> storage shelf port show -fields remote-device,remote-port,connector-state

shelf id connector-state remote-port    remote-device
----- -- --------------- -------------- -----------------
1.1   0  connected       Ethernet1/17/1 CX9332D-cs1
1.1   1  connected       Ethernet1/15/1 CX9364D-cs1
1.1   2  connected       Ethernet1/17/2 CX9332D-cs1
1.1   3  connected       Ethernet1/15/2 CX9364D-cs1
1.1   4  connected       Ethernet1/17/3 CX9332D-cs1
1.1   5  connected       Ethernet1/15/3 CX9364D-cs1
1.1   6  connected       Ethernet1/17/4 CX9332D-cs1
1.1   7  connected       Ethernet1/15/4 CX9364D-cs1
1.1   8  connected       Ethernet1/19/1 CX9332D-cs1
1.1   9  connected       Ethernet1/17/1 CX9364D-cs1
1.1   10 connected       Ethernet1/19/2 CX9332D-cs1
1.1   11 connected       Ethernet1/17/2 CX9364D-cs1
1.1   12 connected       Ethernet1/19/3 CX9332D-cs1
1.1   13 connected       Ethernet1/17/3 CX9364D-cs1
1.1   14 connected       Ethernet1/19/4 CX9332D-cs1
1.1   15 connected       Ethernet1/17/4 CX9364D-cs1

16 entries were displayed.

  4. Verify the connectivity of the remote cluster interfaces.

    1. You can use the network interface check cluster-connectivity show command to display the details of an accessibility check for cluster connectivity:

      network interface check cluster-connectivity show
      Show example 
      cluster1::*> network interface check cluster-connectivity show

                                  Source           Destination     Packet
Node   Date                       LIF              LIF             Loss
------ -------------------------- ---------------- --------------- -----------
node1-01
       6/4/2025 03:13:33 -04:00   node1-01_clus2  node1-02_clus1   none
       6/4/2025 03:13:34 -04:00   node1-01_clus2  node1-02_clus1   none


node1-02
       6/4/2025 03:13:33 -04:00   node1-02_clus2  node1-01_clus1   none
       6/4/2025 03:13:34 -04:00   node1-02_clus2  node1-01_clus2   none
.
.
.

    2. Alternatively, you can use the cluster ping-cluster -node <node-name> command to check the connectivity:

      cluster ping-cluster -node <node-name>

      Show example 
      cluster1::*> cluster ping-cluster -node local
Host is node1-04
Getting addresses from network interface table...
Cluster node1-01_clus1 169.254.36.44 node1-01 e7a
Cluster node1-01_clus2 169.254.7.5   node1-01 e7b
Cluster node1-02_clus1 169.254.197.206 node1-02 e7a
Cluster node1-02_clus2 169.254.195.186 node1-02 e7b
Cluster node1-03_clus1 169.254.192.49 node1-03 e7a
Cluster node1-03_clus2 169.254.182.76 node1-03 e7b
Cluster node1-04_clus1 169.254.59.49 node1-04 e7a
Cluster node1-04_clus2 169.254.62.244 node1-04 e7b
Local = 169.254.59.49 169.254.62.244
Remote = 169.254.36.44 169.254.7.5 169.254.197.206 169.254.195.186 169.254.192.49 169.254.182.76
Cluster Vserver Id = 4294967293
Ping status:
............
Basic connectivity succeeds on 12 path(s)
Basic connectivity fails on 0 path(s)
................................................
Detected 9000 byte MTU on 12 path(s):
    Local 169.254.59.49 to Remote 169.254.182.76
    Local 169.254.59.49 to Remote 169.254.192.49
    Local 169.254.59.49 to Remote 169.254.195.186
    Local 169.254.59.49 to Remote 169.254.197.206
    Local 169.254.59.49 to Remote 169.254.36.44
    Local 169.254.59.49 to Remote 169.254.7.5
    Local 169.254.62.244 to Remote 169.254.182.76
    Local 169.254.62.244 to Remote 169.254.192.49
    Local 169.254.62.244 to Remote 169.254.195.186
    Local 169.254.62.244 to Remote 169.254.197.206
    Local 169.254.62.244 to Remote 169.254.36.44
    Local 169.254.62.244 to Remote 169.254.7.5
Larger than PMTU communication succeeds on 12 path(s)
RPC status:
6 paths up, 0 paths down (tcp check)
6 paths up, 0 paths down (udp check)

  5. Verify that the auto-revert command is enabled on all cluster LIFs:

    network interface show -vserver Cluster -fields auto-revert
    Show example 
    cluster1::*> network interface show -vserver Cluster -fields auto-revert
        Logical
Vserver Interface          Auto-revert
------- ------------------ -----------
Cluster node1-01_clus1     true
Cluster node1-01_clus2     true
Cluster node1-02_clus1     true
Cluster node1-02_clus2     true
Cluster node1-03_clus1     true
Cluster node1-03_clus2     true
Cluster node1-04_clus1     true
Cluster node1-04_clus2     true

8 entries were displayed.
What's next?

After you've prepared to install the NX-OS software and RCF, you install or upgrade the NX-OS software.