Skip to main content
Install and maintain

Upgrade the NX-OS - Cisco Nexus 9332D-GX2B

Contributors netapp-lisa
PDFs

You upgrade the NX-OS on your Cisco Nexus 9332D-GX2B switches to support new hardware or a new ONTAP version on your AFX 1K or AFX 2K system.

Before you begin

  • Make sure you have a current backup of the switch configuration.

  • Verify that your cluster is fully functional, meaning that there are no errors in the logs or similar issues.

About this task

  • If needed, you can refer to the Cisco Nexus 9000 Series NX-OS Software Upgrade and Downgrade Guide for more information.

  • If needed, you can refer to the Cisco Nexus 9000 and 3000 Upgrade and ISSU Matrix, which provides information on In-Service Software Upgrade (ISSU) for Cisco Nexus 9000 Series Switches based on your current and target releases.

    On the page, select Disruptive Upgrade and select your current release and target release from the dropdown list.

  • The version of ONTAP running on your AFX system determines the supported versions of NX-OS for your switches.

    • For switches in an AFX 1K system, running any supported version of ONTAP, NX-OS 10.4.2 and later is supported.

    • For switches in an AFX 2K system, running ONTAP 9.19.1GA or later, NX-OS 10.6 and later is supported.

      For the current list of supported NX-OS versions, see the NetApp Cisco Ethernet Switches page.

      Note Do not upgrade or downgrade your NX-OS version to an unsupported version.

  • Because this procedure requires the use of ONTAP commands and Cisco Nexus 9000 Series Switches commands, the steps indicate which CLI you are using to enter commands.

  • The examples in this procedure use the following switch and node nomenclature:

    • The names of the two Cisco switches are cs1 and cs2.

    • The node names are node1-01, node1-02, node1-03, and node1-04.

    • The cluster LIF names are:

      • node1-01_clus1 and node1-01_clus2 for node1-01

      • node1-02_clus1 and node1-02_clus2 for node1-02

      • node1-03_clus1 and node1-03_clus2 for node1-03

      • node1-04_clus1 and node1-04_clus2 for node1-04

    • The cluster1::*> prompt indicates the name of the cluster.

  • The examples shown in this procedure are for AFX 1K systems with switches running RCF 10.0. If you have an AFX 2K system, the input and output can vary.

Step 1: Prepare to upgrade the NX-OS

Prepare to upgrade the NX-OS by making sure you have a management network connection to each switch, downloading the NX-OS version, checking port status on the cluster, and disabling auto-revert on the cluster LIFs.

Steps

  1. Connect each switch to the management network.

  2. Using the switch CLI, verify connectivity to the server hosting the NX-OS.

    ping <IP address>

    Show example

    This example verifies that the switch can reach the server at IP address 172.19.2.1:

    cs1# ping 172.19.2.1
Pinging 172.19.2.1 with 0 bytes of data:

Reply From 172.19.2.1: icmp_seq = 0. time= 5910 usec.

  3. Download the correct version of NX-OS to the switches from the Cisco software download page.

  4. Using the ONTAP CLI, display the cluster ports on each node that are connected to the cluster switches:

    network device-discovery show
    Show example 
    cluster1::*> network device-discovery show

Node/       Local  Discovered
Protocol    Port   Device (LLDP: ChassisID) Interface         Platform
----------- ------ ------------------------ ----------------  ----------------
node1-01/cdp
            e10a   cs1(FLMXXXXXXXX)         Ethernet1/16/3    N9K-C9332D-GX2B
            e10b   cs2(FDOXXXXXXXX)         Ethernet1/16/3    N9K-C9332D-GX2B
            e11a   cs1(FLMXXXXXXXX)         Ethernet1/16/4    N9K-C9332D-GX2B
            e11b   cs2(FDOXXXXXXXX)         Ethernet1/16/4    N9K-C9332D-GX2B
            e1a    cs1(FLMXXXXXXXX)         Ethernet1/16/1    N9K-C9332D-GX2B
            e1b    cs2(FDOXXXXXXXX)         Ethernet1/16/1    N9K-C9332D-GX2B
            .
            .
            .
            e7a    cs1(FLMXXXXXXXX)         Ethernet1/16/2    N9K-C9332D-GX2B
            e7b    cs2(FDOXXXXXXXX)         Ethernet1/16/2    N9K-C9332D-GX2B

node1-01/lldp
            e10a   cs1 (c8:60:8f:xx:xx:xx)  Ethernet1/16/3    -
            e10b   cs2 (04:e3:87:xx:xx:xx)  Ethernet1/16/3    -
            e11a   cs1 (c8:60:8f:xx:xx:xx)  Ethernet1/16/4    -
            e11b   cs2 (04:e3:87:xx:xx:xx)  Ethernet1/16/4    -
            e1a    cs1 (c8:60:8f:xx:xx:xx)  Ethernet1/16/1    -
            e1b    cs2 (04:e3:87:xx:xx:xx)  Ethernet1/16/1    -
            .
            .
            .
            e7a    cs1 (c8:60:8f:34:78:d3)  Ethernet1/16/2    -
            e7b    cs2 (04:e3:87:19:a2:59)  Ethernet1/16/2    -
.
.
.

  5. Using the ONTAP CLI, check the administrative and operational status of each port:

    1. Cluster ports

      1. Verify that all the cluster ports are up with a healthy status:

        network port show -ipspace Cluster
        Show example 
        cluster1::*> network port show -ipspace Cluster

Node: node1-01
                                                                       Ignore
                                                  Speed(Mbps) Health   Health
Port      IPspace      Broadcast Domain Link MTU  Admin/Oper  Status   Status
--------- ------------ ---------------- ---- ---- ----------- -------- ------
e7a       Cluster      Cluster          up   9000  auto/100000 healthy false
e7b       Cluster      Cluster          up   9000  auto/100000 healthy false

Node: node1-02
                                                                       Ignore
                                                  Speed(Mbps) Health   Health
Port      IPspace      Broadcast Domain Link MTU  Admin/Oper  Status   Status
--------- ------------ ---------------- ---- ---- ----------- -------- ------
e7a       Cluster      Cluster          up   9000  auto/100000 healthy false
e7b       Cluster      Cluster          up   9000  auto/100000 healthy false

Node: node1-03

                                                                       Ignore
                                                  Speed(Mbps) Health   Health
Port      IPspace      Broadcast Domain Link MTU  Admin/Oper  Status   Status
--------- ------------ ---------------- ---- ---- ----------- -------- ------
e7a       Cluster      Cluster          up   9000  auto/10000 healthy  false
e7b       Cluster      Cluster          up   9000  auto/10000 healthy  false

Node: node1-04
                                                                       Ignore
                                                  Speed(Mbps) Health   Health
Port      IPspace      Broadcast Domain Link MTU  Admin/Oper  Status   Status
--------- ------------ ---------------- ---- ---- ----------- -------- ------
e7a       Cluster      Cluster          up   9000  auto/10000 healthy  false
e7b       Cluster      Cluster          up   9000  auto/10000 healthy  false

      2. Verify that all the cluster interfaces (LIFs) are on the home port:

        network interface show -role cluster
        Show example 
        cluster1::*> network interface show -role cluster

            Logical         Status     Network            Current   Current Is
Vserver     Interface       Admin/Oper Address/Mask       Node      Port    Home
----------- --------------- ---------- ------------------ --------- ------- ----
Cluster
            node1-01_clus1  up/up      169.254.36.44/16   node1-01  e7a     true
            node1-01_clus2  up/up      169.254.7.5/16     node1-01  e7b     true
            node1-02_clus1  up/up      169.254.197.206/16 node1-02  e7a     true
            node1-02_clus2  up/up      169.254.195.186/16 node1-02  e7b     true
            node1-03_clus1  up/up      169.254.192.49/16  node1-03  e7a     true
            node1-03_clus2  up/up      169.254.182.76/16  node1-03  e7b     true
            node1-04_clus1  up/up      169.254.59.49/16   node1-04  e7a     true
            node1-04_clus2  up/up      169.254.62.244/16  node1-04  e7b     true

8 entries were displayed.

      3. Verify that the cluster displays information for both cluster switches:

        system cluster-switch show -is-monitoring-enabled-operational true
        Show example 
        cluster1::*> system cluster-switch show -is-monitoring-enabled-operational true

Switch                      Type               Address          Model
--------------------------- ------------------ ---------------- ---------------
cs2(FDOXXXXXXXX)            cluster-network    10.228.137.233   N9K-C9332D-GX2B
     Serial Number: FDOXXXXXXXX
      Is Monitored: true
            Reason: None
  Software Version: Cisco Nexus Operating System (NX-OS) Software, Version
                    10.4(2)
    Version Source: CDP/ISDP


cs1(FLMXXXXXXXX)             cluster-network   10.228.137.253   N9K-C9332D-GX2B
     Serial Number: FLMXXXXXXXX
      Is Monitored: true
            Reason: None
  Software Version: Cisco Nexus Operating System (NX-OS) Software, Version
                    10.4(2)
    Version Source: CDP/ISDP

2 entries were displayed.

    2. HA ports

      1. Verify that all the HA ports are up with a healthy status:

        ha interconnect status show -node <node-name>
        Show example 
        cluster1::*> ha interconnect status show -node node1-01
  (system ha interconnect status show)

                       Node: node1-01
              Link 0 Status: up
              Link 1 Status: up
           Is Link 0 Active: true
           Is Link 1 Active: true
         IC RDMA Connection: up
                       Slot: 0
             Debug Firmware: no


Interconnect Port 0 :
                  Port Name: e1a-17
                        MTU: 4096
           Link Information: ACTIVE


Interconnect Port 1 :
                  Port Name: e1b-18
                        MTU: 4096
           Link Information: ACTIVE

cluster1::*> ha interconnect status show -node node1-02
  (system ha interconnect status show)

                       Node: node1-02
              Link 0 Status: up
              Link 1 Status: up
           Is Link 0 Active: true
           Is Link 1 Active: true
         IC RDMA Connection: up
                       Slot: 0
             Debug Firmware: no


Interconnect Port 0 :
                  Port Name: e1a-17
                        MTU: 4096
           Link Information: ACTIVE


Interconnect Port 1 :
                  Port Name: e1b-18
                        MTU: 4096
           Link Information: ACTIVE
.
.
.

    3. Storage ports

      1. Verify that all the storage ports are up with a healthy status:

        storage port show -port-type ENET
        Show example 
        cluster1::*> storage port show -port-type ENET


                                      Speed
Node               Port Type  Mode    (Gb/s) State    Status
------------------ ---- ----- ------- ------ -------- -----------
node1-01
                   e10a ENET  -          100 enabled  online
                   e10b ENET  -          100 enabled  online
                   e11a ENET  -          100 enabled  online
                   e11b ENET  -          100 enabled  online
node1-02
                   e10a ENET  -          100 enabled  online
                   e10b ENET  -          100 enabled  online
                   e11a ENET  -          100 enabled  online
                   e11b ENET  -          100 enabled  online
node1-03
                   e10a ENET  -          100 enabled  online
                   e10b ENET  -          100 enabled  online
                   e11a ENET  -          100 enabled  online
node1-04
                   e10a ENET  -          100 enabled  online
                   e10b ENET  -          100 enabled  online
                   e11a ENET  -          100 enabled  online
                   e11b ENET  -          100 enabled  online
16 entries were displayed.

    4. Storage shelf ports

      1. Verify that all the storage shelf ports are up with a healthy status:

        storage shelf port show
        Show example 
        cluster1::*> storage shelf port show

Shelf ID Module State        Internal?
----- -- ------ ------------ ---------
1.1
       0 A      connected    false
       1 A      connected    false
       2 A      connected    false
       3 A      connected    false
       4 A      connected    false
       5 A      connected    false
       6 A      connected    false
       7 A      connected    false
       8 B      connected    false
       9 B      connected    false
      10 B      connected    false
      11 B      connected    false
      12 B      connected    false
      13 B      connected    false
      14 B      connected    false
      15 B      connected    false

16 entries were displayed.

      2. Verify the connection status of all the storage shelf ports:

        storage shelf port show -fields remote-device,remote-port,connector-state
        Show example 
        cluster1::*> storage shelf port show -fields remote-device,remote-port,connector-state

shelf id connector-state remote-port    remote-device
----- -- --------------- -------------- -----------------
1.1   0  connected       Ethernet1/17/1 CX9332D-cs1
1.1   1  connected       Ethernet1/15/1 CX9364D-cs1
1.1   2  connected       Ethernet1/17/2 CX9332D-cs1
1.1   3  connected       Ethernet1/15/2 CX9364D-cs1
1.1   4  connected       Ethernet1/17/3 CX9332D-cs1
1.1   5  connected       Ethernet1/15/3 CX9364D-cs1
1.1   6  connected       Ethernet1/17/4 CX9332D-cs1
1.1   7  connected       Ethernet1/15/4 CX9364D-cs1
1.1   8  connected       Ethernet1/19/1 CX9332D-cs1
1.1   9  connected       Ethernet1/17/1 CX9364D-cs1
1.1   10 connected       Ethernet1/19/2 CX9332D-cs1
1.1   11 connected       Ethernet1/17/2 CX9364D-cs1
1.1   12 connected       Ethernet1/19/3 CX9332D-cs1
1.1   13 connected       Ethernet1/17/3 CX9364D-cs1
1.1   14 connected       Ethernet1/19/4 CX9332D-cs1
1.1   15 connected       Ethernet1/17/4 CX9364D-cs1

16 entries were displayed.

  6. Using the ONTAP CLI, disable auto-revert on the cluster LIFs. The cluster LIFs fail over to the partner cluster switch and remain there as you perform this procedure on the targeted switch:

    network interface modify -vserver Cluster -lif * -auto-revert false

Step 2: Install the NX-OS

You install the NX-OS on the first switch (cs1) by installing and verifying the new versions of the NX-OS and EPLD, verifying the health of all ports on the cluster, and then repeat these steps on the second switch (cs2).

  1. Using the switch CLI, copy NX-OS and EPLD images to a switch.

    You use this command two times: the first time is to copy the NX-OS version and the second time is to copy the EPLD version.

    copy <protocol>: bootflash: vrf management

    Show example

    The example uses sftp.

    cs1# copy sftp: bootflash: vrf management
Enter source filename: /code/nxos.10.4.2.bin
Enter hostname for the sftp server: 172.19.2.1
Enter username: root

Outbound-ReKey for 172.19.2.1:22
Inbound-ReKey for 172.19.2.1:22
root@172.19.2.1's password:
sftp> progress
Progress meter enabled
sftp> get   /code/nxos.10.4.2.bin  /bootflash/nxos.10.4.2.bin
/code/nxos.10.4.2.bin  100% 1261MB   9.3MB/s   02:15
sftp> exit
Copy complete, now saving to disk (please wait)...
Copy complete.

cs1# copy sftp: bootflash: vrf management
Enter source filename: /code/n9000-epld.10.4.2.F.img
Enter hostname for the sftp server: 172.19.2.1
Enter username: user1

Outbound-ReKey for 172.19.2.1:22
Inbound-ReKey for 172.19.2.1:22
user1@172.19.2.1's password:
sftp> progress
Progress meter enabled
sftp> get   /code/n9000-epld.10.4.2.F.img  /bootflash/n9000-epld.10.4.2.F.img
/code/n9000-epld.10.4.2.F.img  100%  161MB   9.5MB/s   00:16
sftp> exit
Copy complete, now saving to disk (please wait)...
Copy complete.

  2. Using the switch CLI, verify the running version of NX-OS:

    show version

    Show example 
    cs1# show version
Cisco Nexus Operating System (NX-OS) Software
TAC support: http://www.cisco.com/tac
Copyright (C) 2002-2025, Cisco and/or its affiliates.
All rights reserved.
The copyrights to certain works contained in this software are
owned by other third parties and used and distributed under their own
licenses, such as open source.  This software is provided "as is," and unless
otherwise stated, there is no warranty, express or implied, including but not
limited to warranties of merchantability and fitness for a particular purpose.
Certain components of this software are licensed under
the GNU General Public License (GPL) version 2.0 or
GNU General Public License (GPL) version 3.0  or the GNU
Lesser General Public License (LGPL) Version 2.1 or
Lesser General Public License (LGPL) Version 2.0.
A copy of each such license is available at
http://www.opensource.org/licenses/gpl-2.0.php and
http://opensource.org/licenses/gpl-3.0.html and
http://www.opensource.org/licenses/lgpl-2.1.php and
http://www.gnu.org/licenses/old-licenses/library.txt.


Software
  BIOS: version 01.14
  NXOS: version 10.4(1) [Feature Release]
  Host NXOS: version 10.4(1)
  BIOS compile time:  11/25/2024
  NXOS image file is: bootflash:///nxos64-cs.10.4.1.F.bin
  NXOS compile time:  11/30/2023 12:00:00 [12/14/2023 05:25:50]
  NXOS boot mode: LXC


Hardware
  cisco Nexus9000 C9332D-GX2B Chassis
  Intel(R) Xeon(R) CPU D-1633N @ 2.50GHz with 32802156 kB of memory.
  Processor Board ID FLMXXXXXXXX
  Device name: cs2
  bootflash:  115802886 kB


Kernel uptime is 5 day(s), 2 hour(s), 13 minute(s), 21 second(s)


Last reset at 3580 usecs after Thu Jun  5 15:55:08 2025
  Reason: Reset Requested by CLI command reload
  System version: 10.4(1)
  Service:


plugin
  Core Plugin, Ethernet Plugin


Active Package(s):

cs1#

  3. Using the switch CLI, install the NX-OS image.

    Installing the image file causes it to be loaded every time the switch is rebooted.

    install all nxos bootflash:nxos.<version number>.bin

    Show example

    The example uses NX-OS version 10.4.2.

    cs1# install all nxos bootflash:nxos.10.4.2.bin

Installer will perform compatibility check first. Please wait.
Installer is forced disruptive

Verifying image bootflash:/nxos.10.4.2.bin for boot variable "nxos".
[] 100% -- SUCCESS

Verifying image type.
[] 100% -- SUCCESS

Preparing "nxos" version info using image bootflash:/nxos.10.4.2.bin.
[] 100% -- SUCCESS

Preparing "bios" version info using image bootflash:/nxos.10.4.2.bin.
[] 100% -- SUCCESS

Performing module support checks.
[] 100% -- SUCCESS

Notifying services about system upgrade.
[] 100% -- SUCCESS


Compatibility check is done:
Module  Bootable  Impact          Install-type  Reason
------  --------  --------------- ------------  ---------
  1     yes       Disruptive      Reset         Default upgrade is not hitless



Images will be upgraded according to following table:

Module   Image    Running-Version(pri:alt)                 New-Version         Upg-Required
------- --------- ---------------------------------------- ------------------- ------------
  1      nxos     10.4(1)                                  10.4(2)             Yes
  1      bios     xx.xx.:xx.xx                             xxx                 No


Switch will be reloaded for disruptive upgrade.

Do you want to continue with the installation (y/n)? [n] y

Install is in progress, please wait.

Performing runtime checks.
[] 100% -- SUCCESS

Setting boot variables.
[] 100% -- SUCCESS

Performing configuration copy.
[] 100% -- SUCCESS

Module 1: Refreshing compact flash and upgrading bios/loader/bootrom.
Warning: please do not remove or power off the module at this time.
[] 100% -- SUCCESS

Finishing the upgrade, switch will reboot in 10 seconds.

  4. Using the switch CLI, verify the new version of NX-OS after the switch has rebooted:

    show version

    Show example 
    cs1# show version
Cisco Nexus Operating System (NX-OS) Software
TAC support: http://www.cisco.com/tac
Copyright (C) 2002-2025, Cisco and/or its affiliates.
All rights reserved.
The copyrights to certain works contained in this software are
owned by other third parties and used and distributed under their own
licenses, such as open source.  This software is provided "as is," and unless
otherwise stated, there is no warranty, express or implied, including but not
limited to warranties of merchantability and fitness for a particular purpose.
Certain components of this software are licensed under
the GNU General Public License (GPL) version 2.0 or
GNU General Public License (GPL) version 3.0  or the GNU
Lesser General Public License (LGPL) Version 2.1 or
Lesser General Public License (LGPL) Version 2.0.
A copy of each such license is available at
http://www.opensource.org/licenses/gpl-2.0.php and
http://opensource.org/licenses/gpl-3.0.html and
http://www.opensource.org/licenses/lgpl-2.1.php and
http://www.gnu.org/licenses/old-licenses/library.txt.


Software
  BIOS: version 01.14
  NXOS: version 10.4(2) [Feature Release]
  Host NXOS: version 10.4(2)
  BIOS compile time:  11/25/2024
  NXOS image file is: bootflash:///nxos64-cs.10.4.2.F.bin
  NXOS compile time:  11/30/2023 12:00:00 [12/14/2023 05:25:50]
  NXOS boot mode: LXC


Hardware
  cisco Nexus9000 C9332D-GX2B Chassis
  Intel(R) Xeon(R) CPU D-1633N @ 2.50GHz with 32802156 kB of memory.
  Processor Board ID FLMXXXXXXXX
  Device name: cs2
  bootflash:  115802886 kB


Kernel uptime is 5 day(s), 2 hour(s), 13 minute(s), 21 second(s)


Last reset at 3580 usecs after Thu Jun  5 15:55:08 2025
  Reason: Reset Requested by CLI command reload
  System version: 10.4(2)
  Service:


plugin
  Core Plugin, Ethernet Plugin


Active Package(s):

cs1#

  5. Using the switch CLI, upgrade the EPLD image using the following commands:

    show version module 1 epld

    install epld bootflash:n9000-epld.<version>.img module all

    After the last command is executed, the switch reboots.

    Show example

    The example uses EPLD version 10.4.2.F

    cs1# show version module 1 epld

EPLD Device                     Version
---------------------------------------
MI   FPGA                        0x7
IO   FPGA                        0x17
MI   FPGA2                       0x2
GEM  FPGA                        0x2
GEM  FPGA                        0x2
GEM  FPGA                        0x2
GEM  FPGA                        0x2

cs1# install epld bootflash:n9000-epld.10.4.2.F.img module all
Compatibility check:
Module        Type         Upgradable        Impact    Reason
------  ------------------ ----------------- --------- -----------
     1         SUP         Yes       disruptive  Module Upgradable

Retrieving EPLD versions.... Please wait.
Images will be upgraded according to following table:
Module  Type   EPLD              Running-Version   New-Version  Upg-Required
------- ------ ----------------- ----------------- ------------ ------------
     1  SUP    MI FPGA           0x07              0x07         No
     1  SUP    IO FPGA           0x17              0x19         Yes
     1  SUP    MI FPGA2          0x02              0x02         No
The above modules require upgrade.
The switch will be reloaded at the end of the upgrade
Do you want to continue (y/n) ?  [n] y

Proceeding to upgrade Modules.

Starting Module 1 EPLD Upgrade

Module 1 : IO FPGA [Programming] : 100.00% (     64 of      64 sectors)
Module 1 EPLD upgrade is successful.
Module   Type  Upgrade-Result
-------- ----- --------------
     1   SUP   Success

EPLDs upgraded.

Module 1 EPLD upgrade is successful.

  6. After the switch reboots, use the switch CLI to log in and verify that the new version of EPLD loaded successfully.

    show version module 1 epld

    Show example 
    cs1# show version module 1 epld

EPLD Device                     Version
---------------------------------------
MI   FPGA                        0x7
IO   FPGA                        0x19
MI   FPGA2                       0x2
GEM  FPGA                        0x2
GEM  FPGA                        0x2
GEM  FPGA                        0x2
GEM  FPGA                        0x2

  7. Using the ONTAP CLI, verify the health of all the ports on the cluster.

    1. Cluster ports

      1. Verify that cluster ports are up and healthy across all nodes in the cluster:

        network port show -ipspace Cluster
        Show example 
        cluster1::*> network port show -ipspace Cluster

Node: node1-01
                                                                       Ignore
                                                  Speed(Mbps) Health   Health
Port      IPspace      Broadcast Domain Link MTU  Admin/Oper  Status   Status
--------- ------------ ---------------- ---- ---- ----------- -------- ------
e7a       Cluster      Cluster          up   9000  auto/10000 healthy  false
e7b       Cluster      Cluster          up   9000  auto/10000 healthy  false

Node: node1-02
                                                                       Ignore
                                                  Speed(Mbps) Health   Health
Port      IPspace      Broadcast Domain Link MTU  Admin/Oper  Status   Status
--------- ------------ ---------------- ---- ---- ----------- -------- ------
e7a       Cluster      Cluster          up   9000  auto/10000 healthy  false
e7b       Cluster      Cluster          up   9000  auto/10000 healthy  false

Node: node1-03
                                                                       Ignore
                                                  Speed(Mbps) Health   Health
Port      IPspace      Broadcast Domain Link MTU  Admin/Oper  Status   Status
--------- ------------ ---------------- ---- ---- ----------- -------- ------
e7a       Cluster      Cluster          up   9000  auto/100000 healthy false
e7b       Cluster      Cluster          up   9000  auto/100000 healthy false

Node: node1-04
                                                                       Ignore
                                                  Speed(Mbps) Health   Health
Port      IPspace      Broadcast Domain Link MTU  Admin/Oper  Status   Status
--------- ------------ ---------------- ---- ---- ----------- -------- ------
e7a       Cluster      Cluster          up   9000  auto/100000 healthy false
e7b       Cluster      Cluster          up   9000  auto/100000 healthy false

      2. Verify the switch health from the cluster:

        network device-discovery show -protocol cdp
        system cluster-switch show -is-monitoring-enabled-operational true
        Show example 
        cluster1::*> network device-discovery show -protocol cdp

node1-02/cdp
            e10a   cs1(FLMXXXXXXXX)         Ethernet1/16/3    N9K-C9332D-GX2B
            e10b   cs2(FDOXXXXXXXX)         Ethernet1/16/3    N9K-C9332D-GX2B
            e11a   cs1(FLMXXXXXXXX)         Ethernet1/16/4    N9K-C9332D-GX2B
            e11b   cs2(FDOXXXXXXXX)         Ethernet1/16/4    N9K-C9332D-GX2B
            e1a    cs1(FLMXXXXXXXX)         Ethernet1/16/1    N9K-C9332D-GX2B
            e1b    cs2(FDOXXXXXXXX)         Ethernet1/16/1    N9K-C9332D-GX2B
            .
            .
            .
            e7a    cs1(FLM284504N6)         Ethernet1/16/2    N9K-C9332D-GX2B
            e7b    cs2(FDO2846056X)         Ethernet1/16/2    N9K-C9332D-GX2B

node1-01/cdp
            e10a   cs1(FLMXXXXXXXX)         Ethernet1/16/3    N9K-C9332D-GX2B
            e10b   cs2(FDOXXXXXXXX)         Ethernet1/16/3    N9K-C9332D-GX2B
            e11a   cs1(FLMXXXXXXXX)         Ethernet1/16/4    N9K-C9332D-GX2B
            e11b   cs2(FDOXXXXXXXX)         Ethernet1/16/4    N9K-C9332D-GX2B
            e1a    cs1(FLMXXXXXXXX)         Ethernet1/16/1    N9K-C9332D-GX2B
            e1b    cs2(FDOXXXXXXXX)         Ethernet1/16/1    N9K-C9332D-GX2B
           .
           .
           .
            e7a    cs1(FLMXXXXXXXX)         Ethernet1/16/2    N9K-C9332D-GX2B
            e7b    cs2(FDOXXXXXXXX)         Ethernet1/16/2    N9K-C9332D-GX2B
.
.
.

cluster1::*> system cluster-switch show -is-monitoring-enabled-operational true
Switch                      Type               Address          Model
--------------------------- ------------------ ---------------- ---------------
cs1(FDOXXXXXXXX)            cluster-network    10.228.137.233   N9K-C9332D-GX2B
     Serial Number: FDOXXXXXXXX
      Is Monitored: true
            Reason: None
  Software Version: Cisco Nexus Operating System (NX-OS) Software, Version
                    10.4(2)
    Version Source: CDP/ISDP


cs1(FLMXXXXXXXX)             cluster-network   10.228.137.253   N9K-C9332D-GX2B
     Serial Number: FLMXXXXXXXX
      Is Monitored: true
            Reason: None
  Software Version: Cisco Nexus Operating System (NX-OS) Software, Version
                    10.4(2)
    Version Source: CDP/ISDP

    2. HA ports

      1. Verify that all the HA ports are up with a healthy status:

        ha interconnect status show -node <node-name>
        Show example 
        cluster1::*> ha interconnect status show -node node1-01
  (system ha interconnect status show)

                       Node: node1-01
              Link 0 Status: up
              Link 1 Status: up
           Is Link 0 Active: true
           Is Link 1 Active: true
         IC RDMA Connection: up
                       Slot: 0
             Debug Firmware: no


Interconnect Port 0 :
                  Port Name: e1a-17
                        MTU: 4096
           Link Information: ACTIVE


Interconnect Port 1 :
                  Port Name: e1b-18
                        MTU: 4096
           Link Information: ACTIVE

cluster1::*> ha interconnect status show -node node1-02
  (system ha interconnect status show)

                       Node: node1-02
              Link 0 Status: up
              Link 1 Status: up
           Is Link 0 Active: true
           Is Link 1 Active: true
         IC RDMA Connection: up
                       Slot: 0
             Debug Firmware: no


Interconnect Port 0 :
                  Port Name: e1a-17
                        MTU: 4096
           Link Information: ACTIVE


Interconnect Port 1 :
                  Port Name: e1b-18
                        MTU: 4096
           Link Information: ACTIVE
.
.
.

    3. Storage ports

      1. Verify that all the storage ports are up with a healthy status:

        storage port show -port-type ENET
        Show example 
        cluster1::*> storage port show -port-type ENET


                                      Speed
Node               Port Type  Mode    (Gb/s) State    Status
------------------ ---- ----- ------- ------ -------- -----------
node1-01
                   e10a ENET  -          100 enabled  online
                   e10b ENET  -          100 enabled  online
                   e11a ENET  -          100 enabled  online
                   e11b ENET  -          100 enabled  online
node1-02
                   e10a ENET  -          100 enabled  online
                   e10b ENET  -          100 enabled  online
                   e11a ENET  -          100 enabled  online
                   e11b ENET  -          100 enabled  online
node1-03
                   e10a ENET  -          100 enabled  online
                   e10b ENET  -          100 enabled  online
                   e11a ENET  -          100 enabled  online
node1-04
                   e10a ENET  -          100 enabled  online
                   e10b ENET  -          100 enabled  online
                   e11a ENET  -          100 enabled  online
                   e11b ENET  -          100 enabled  online
16 entries were displayed.

    4. Storage shelf ports

      1. Verify that all the storage shelf ports are up with a healthy status:

        storage shelf port show
        Show example 
        cluster1::*> storage shelf port show

Shelf ID Module State        Internal?
----- -- ------ ------------ ---------
1.1
       0 A      connected    false
       1 A      connected    false
       2 A      connected    false
       3 A      connected    false
       4 A      connected    false
       5 A      connected    false
       6 A      connected    false
       7 A      connected    false
       8 B      connected    false
       9 B      connected    false
      10 B      connected    false
      11 B      connected    false
      12 B      connected    false
      13 B      connected    false
      14 B      connected    false
      15 B      connected    false

16 entries were displayed.

      2. Verify the connection status of all the storage shelf ports:

        storage shelf port show -fields remote-device,remote-port,connector-state
        Show example 
        cluster1::*> storage shelf port show -fields remote-device,remote-port,connector-state

shelf id connector-state remote-port    remote-device
----- -- --------------- -------------- -----------------
1.1   0  connected       Ethernet1/17/1 CX9332D-cs1
1.1   1  connected       Ethernet1/15/1 CX9364D-cs1
1.1   2  connected       Ethernet1/17/2 CX9332D-cs1
1.1   3  connected       Ethernet1/15/2 CX9364D-cs1
1.1   4  connected       Ethernet1/17/3 CX9332D-cs1
1.1   5  connected       Ethernet1/15/3 CX9364D-cs1
1.1   6  connected       Ethernet1/17/4 CX9332D-cs1
1.1   7  connected       Ethernet1/15/4 CX9364D-cs1
1.1   8  connected       Ethernet1/19/1 CX9332D-cs1
1.1   9  connected       Ethernet1/17/1 CX9364D-cs1
1.1   10 connected       Ethernet1/19/2 CX9332D-cs1
1.1   11 connected       Ethernet1/17/2 CX9364D-cs1
1.1   12 connected       Ethernet1/19/3 CX9332D-cs1
1.1   13 connected       Ethernet1/17/3 CX9364D-cs1
1.1   14 connected       Ethernet1/19/4 CX9332D-cs1
1.1   15 connected       Ethernet1/17/4 CX9364D-cs1

16 entries were displayed.

  8. Using the ONTAP CLI, verify that the cluster is healthy:

    cluster show
    Show example 
    cluster1::*> cluster show


Node                 Health  Eligibility   Epsilon
-------------------- ------- ------------  ------------
node1-01             true    true          false
node1-02             true    true          false
node1-03             true    true          false
node1-04             true    true          true

4 entries were displayed.

  9. Repeat these steps to install NX-OS and EPLD images on switch cs2.

Step 3: Complete the NX-OS upgrade

Using the ONTAP CLI, complete the NX-OS upgrade by re-enabling auto-revert on the cluster LIFs and verifying the cluster LIFs reverted to their home ports.

  1. Re-enable auto-revert on the cluster LIFs.

    network interface modify -vserver Cluster -lif * -auto-revert true

  2. Verify that the cluster LIFs have reverted to their home port:

    network interface show -role cluster
    Show example 
    cluster1::*> network interface show -role cluster

            Logical         Status     Network            Current     Current Is
Vserver     Interface       Admin/Oper Address/Mask       Node        Port    Home
----------- --------------- ---------- ------------------ ----------- ------- ----
Cluster
            node1-01_clus1  up/up      169.254.36.44/16   node1-01    e7a     true
            node1-01_clus2  up/up      169.254.7.5/16     node1-01    e7b     true
            node1-02_clus1  up/up      169.254.197.206/16 node1-02    e7a     true
            node1-02_clus2  up/up      169.254.195.186/16 node1-02    e7b     true
            node1-03_clus1  up/up      169.254.192.49/16  node1-03    e7a     true
            node1-03_clus2  up/up      169.254.182.76/16  node1-03    e7b     true
            node1-04_clus1  up/up      169.254.59.49/16   node1-04    e7a     true
            node1-04_clus2  up/up      169.254.62.244/16  node1-04    e7b     true

    If any cluster LIFs have not returned to their home ports, revert them manually from the local node:

    network interface revert -vserver Cluster -lif <lif-name>
What's next?

After you've upgraded the NX-OS on your switches, you upgrade the RCF starting with the Install or upgrade the RCF overview.