Install or upgrade the NX-OS software
Follow this procedure to install or upgrade the NX-OS software on the Nexus 9336C-FX2 and 9336C-FX2-T cluster switches.
Before you begin, complete the procedure in Prepare to install NX-OS and RCF.
Review requirements
Make sure you do the following:
-
Run the
show install all impact nxos bootflash:<image_name>.bincommand on the switch to review the impact of installing or upgrading the new NX-OS software image. It verifies the image integrity, checks for necessary reboots, evaluates hardware compatibility, and confirms sufficient space. -
Review the release notes for the target NX-OS software version to check for any specific requirements.
-
Verify that you have a current backup of the switch configuration.
-
Verify that you have a fully functioning cluster (no errors in the logs or similar issues).
-
Consult the switch compatibility table for the supported ONTAP and NX-OS versions.
-
Software Upgrade and downgrade guides
Refer to the Cisco website for software upgrade and downgrade guides.
-
Cisco Nexus 9000 and 3000 Upgrade and ISSU Matrix
Provides information on Disruptive Upgrade/Downgrade for Cisco NX-OS software on Nexus 9000 Series Switches based on your current and target releases.
On the page, select Disruptive Upgrade and select your current release and target release from the dropdown list.
The examples in this procedure use the following switch and node nomenclature:
-
The names of the two Cisco switches are cs1 and cs2.
-
The node names are cluster1-01, cluster1-02, cluster1-03, and cluster1-04.
-
The cluster LIF names are cluster1-01_clus1, cluster1-01_clus2, cluster1-02_clus1, cluster1-02_clus2 , cluster1-03_clus1, cluster1-03_clus2, cluster1-04_clus1, and cluster1-04_clus2.
-
The
cluster1::*>prompt indicates the name of the cluster.
Install the software
Use ONTAP commands in this procedure unless a Cisco Nexus 9000 Series Switch command is specified.
-
Connect the cluster switch to the management network.
-
Use the ping command to verify connectivity to the server hosting the NX-OS software and the RCF.
ping <IP address> vrf managementShow example
This example verifies that the switch can reach the server at IP address 172.19.2.1:
cs2# ping 172.19.2.1 vrf management Pinging 172.19.2.1 with 0 bytes of data: Reply From 172.19.2.1: icmp_seq = 0. time= 5910 usec.
-
Display the cluster ports on each node that are connected to the cluster switches:
network device-discovery showShow example
cluster1::*> network device-discovery show Node/ Local Discovered Protocol Port Device (LLDP: ChassisID) Interface Platform ----------- ------ ------------------------- ----------------- -------------- cluster1-01/cdp e0a cs1 Ethernet1/7 N9K-C9336C-FX2 e0b cs2 Ethernet1/7 N9K-C9336C-FX2 cluster1-02/cdp e0a cs1 Ethernet1/8 N9K-C9336C-FX2 e0b cs2 Ethernet1/8 N9K-C9336C-FX2 cluster1-03/cdp e0a cs1 Ethernet1/1/1 N9K-C9336C-FX2 e0b cs2 Ethernet1/1/1 N9K-C9336C-FX2 cluster1-04/cdp e0a cs1 Ethernet1/1/2 N9K-C9336C-FX2 e0b cs2 Ethernet1/1/2 N9K-C9336C-FX2 cluster1::*> -
Check the administrative and operational status of each cluster port.
-
Verify that all the cluster ports are up with a healthy status:
network port show -ipspace ClusterShow example
cluster1::*> network port show -ipspace Cluster Node: cluster1-01 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ------------ -------- ------ e0a Cluster Cluster up 9000 auto/100000 healthy false e0b Cluster Cluster up 9000 auto/100000 healthy false Node: cluster1-02 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ------------ -------- ------ e0a Cluster Cluster up 9000 auto/100000 healthy false e0b Cluster Cluster up 9000 auto/100000 healthy false Node: cluster1-03 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ------------ -------- ------ e0a Cluster Cluster up 9000 auto/100000 healthy false e0b Cluster Cluster up 9000 auto/100000 healthy false Node: cluster1-04 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ------------ -------- ------ e0a Cluster Cluster up 9000 auto/100000 healthy false e0b Cluster Cluster up 9000 auto/100000 healthy false cluster1::*> -
Verify that all the cluster interfaces (LIFs) are on the home port:
network interface show -vserver ClusterShow example
cluster1::*> network interface show -vserver Cluster Logical Status Network Current Current Is Vserver Interface Admin/Oper Address/Mask Node Port Home ----------- ------------------ ---------- ------------------ ------------------- ------- ---- Cluster cluster1-01_clus1 up/up 169.254.3.4/23 cluster1-01 e0b true cluster1-01_clus2 up/up 169.254.3.5/23 cluster1-01 e0b true cluster1-02_clus1 up/up 169.254.3.8/23 cluster1-02 e0b true cluster1-02_clus2 up/up 169.254.3.9/23 cluster1-02 e0b true cluster1-03_clus1 up/up 169.254.1.3/23 cluster1-03 e0b true cluster1-03_clus2 up/up 169.254.1.1/23 cluster1-03 e0b true cluster1-04_clus1 up/up 169.254.1.6/23 cluster1-04 e0b true cluster1-04_clus2 up/up 169.254.1.7/23 cluster1-04 e0b true cluster1::*> -
Verify that the cluster displays information for both cluster switches:
system cluster-switch show -is-monitoring-enabled-operational trueShow example
cluster1::*> system cluster-switch show -is-monitoring-enabled-operational true Switch Type Address Model --------------------------- ------------------ ---------------- -------------- cs1 cluster-network 10.233.205.90 N9K-C9336C-FX2 Serial Number: FOCXXXXXXGD Is Monitored: true Reason: None Software Version: Cisco Nexus Operating System (NX-OS) Software, Version 9.3(5) Version Source: CDP cs2 cluster-network 10.233.205.91 N9K-C9336C-FX2 Serial Number: FOCXXXXXXGS Is Monitored: true Reason: None Software Version: Cisco Nexus Operating System (NX-OS) Software, Version 9.3(5) Version Source: CDP cluster1::*>
-
-
Disable auto-revert on the cluster LIFs. The cluster LIFs fail over to the partner cluster switch and remain there as you perform the upgrade procedure on the targeted switch:
network interface modify -vserver Cluster -lif * -auto-revert false -
Download and install the NX-OS software and EPLD image. The steps can vary based on your current NX-OS version and the target NX-OS version.
Choose the tab that applies to your situation:
-
Upgrade to 10.4(5)(M)
-
Upgrade from 10.4(5)(M) or earlier to 10.5(3)(F) or later
-
Upgrade from 10.5(3)(F) to a later version
-
-
Copy the NX-OS software and EPLD images to the Nexus 9336C-FX2 switch.
copy <protocol>: bootflash: vrf managementExample
cs2# copy sftp: bootflash: vrf management Enter source filename: /code/nxos.10.4.5.M.bin Enter hostname for the sftp server: 172.19.2.1 Enter username: user1 Outbound-ReKey for 172.19.2.1:22 Inbound-ReKey for 172.19.2.1:22 user1@172.19.2.1's password: sftp> progress Progress meter enabled sftp> get /code/nxos.10.4.5.M.bin /bootflash/nxos.10.4.5.M.bin /code/nxos.10.4.5.M.bin 100% 1261MB 9.3MB/s 02:15 sftp> exit Copy complete, now saving to disk (please wait)... Copy complete. cs2# copy sftp: bootflash: vrf management Enter source filename: /code/n9000-epld.10.4.5.M.img Enter hostname for the sftp server: 172.19.2.1 Enter username: user1 Outbound-ReKey for 172.19.2.1:22 Inbound-ReKey for 172.19.2.1:22 user1@172.19.2.1's password: sftp> progress Progress meter enabled sftp> get /code/n9000-epld.10.4.5.M.img /bootflash/n9000-epld.10.4.5.M.img /code/n9000-epld.10.4.5.M.img 100% 161MB 9.5MB/s 00:16 sftp> exit Copy complete, now saving to disk (please wait)... Copy complete.
-
Verify the running version of the NX-OS software:
show versionExample
cs2# show version Cisco Nexus Operating System (NX-OS) Software TAC support: http://www.cisco.com/tac Copyright (C) 2002-2020, Cisco and/or its affiliates. All rights reserved. The copyrights to certain works contained in this software are owned by other third parties and used and distributed under their own licenses, such as open source. This software is provided "as is," and unless otherwise stated, there is no warranty, express or implied, including but not limited to warranties of merchantability and fitness for a particular purpose. Certain components of this software are licensed under the GNU General Public License (GPL) version 2.0 or GNU General Public License (GPL) version 3.0 or the GNU Lesser General Public License (LGPL) Version 2.1 or Lesser General Public License (LGPL) Version 2.0. A copy of each such license is available at http://www.opensource.org/licenses/gpl-2.0.php and http://opensource.org/licenses/gpl-3.0.html and http://www.opensource.org/licenses/lgpl-2.1.php and http://www.gnu.org/licenses/old-licenses/library.txt. Software BIOS: version 08.38 NXOS: version 10.3(5) BIOS compile time: 05/29/2020 NXOS image file is: bootflash:///nxos.10.3.5.bin NXOS compile time: 4/28/2020 21:00:00 [04/29/2020 02:28:31] Hardware cisco Nexus9000 C9336C-FX2 Chassis Intel(R) Xeon(R) CPU E5-2403 v2 @ 1.80GHz with 8154432 kB of memory. Processor Board ID FOC20291J6K Device name: cs2 bootflash: 53298520 kB Kernel uptime is 0 day(s), 0 hour(s), 3 minute(s), 42 second(s) Last reset at 157524 usecs after Mon Nov 2 18:32:06 2020 Reason: Reset Requested by CLI command reload System version: 9.3(4) Service: plugin Core Plugin, Ethernet Plugin Active Package(s): cs2#
-
Install the NX-OS image.
Installing the image file causes it to be loaded every time the switch is rebooted.
install all nxos bootflash:<bin_file_name>Example
cs2# install all nxos bootflash:nxos.10.4.5.M.bin Installer will perform compatibility check first. Please wait. Installer is forced disruptive Verifying image bootflash:/nxos.10.4.5.M.bin for boot variable "nxos". [] 100% -- SUCCESS Verifying image type. [] 100% -- SUCCESS Preparing "nxos" version info using image bootflash:/nxos.10.4.5.M.bin. [] 100% -- SUCCESS Preparing "bios" version info using image bootflash:/nxos.10.4.5.M.bin. [] 100% -- SUCCESS Performing module support checks. [] 100% -- SUCCESS Notifying services about system upgrade. [] 100% -- SUCCESS Compatibility check is done: Module Bootable Impact Install-type Reason ------ -------- --------------- ------------ --------- 1 yes Disruptive Reset Default upgrade is not hitless Images will be upgraded according to following table: Module Image Running-Version(pri:alt) New-Version Upg-Required ------- --------- ---------------------------------------- ------------------- ------------ 1 nxos 10.3(5) 10.4.5(M) yes 1 bios v08.37(01/28/2020):v08.23(09/23/2015) v08.38(05/29/2020) yes Switch will be reloaded for disruptive upgrade. Do you want to continue with the installation (y/n)? [n] y Install is in progress, please wait. Performing runtime checks. [] 100% -- SUCCESS Setting boot variables. [] 100% -- SUCCESS Performing configuration copy. [] 100% -- SUCCESS Module 1: Refreshing compact flash and upgrading bios/loader/bootrom. Warning: please do not remove or power off the module at this time. [] 100% -- SUCCESS Finishing the upgrade, switch will reboot in 10 seconds.
-
Verify the new version of NX-OS software after the switch has rebooted:
show versionExample
cs2# show version Cisco Nexus Operating System (NX-OS) Software TAC support: http://www.cisco.com/tac Copyright (C) 2002-2020, Cisco and/or its affiliates. All rights reserved. The copyrights to certain works contained in this software are owned by other third parties and used and distributed under their own licenses, such as open source. This software is provided "as is," and unless otherwise stated, there is no warranty, express or implied, including but not limited to warranties of merchantability and fitness for a particular purpose. Certain components of this software are licensed under the GNU General Public License (GPL) version 2.0 or GNU General Public License (GPL) version 3.0 or the GNU Lesser General Public License (LGPL) Version 2.1 or Lesser General Public License (LGPL) Version 2.0. A copy of each such license is available at http://www.opensource.org/licenses/gpl-2.0.php and http://opensource.org/licenses/gpl-3.0.html and http://www.opensource.org/licenses/lgpl-2.1.php and http://www.gnu.org/licenses/old-licenses/library.txt. Software BIOS: version 05.33 NXOS: version 10.4.5(M) BIOS compile time: 09/08/2018 NXOS image file is: bootflash:///nxos.10.4.5.M.bin NXOS compile time: 11/4/2018 21:00:00 [11/05/2018 06:11:06] Hardware cisco Nexus9000 C9336C-FX2 Chassis Intel(R) Xeon(R) CPU E5-2403 v2 @ 1.80GHz with 8154432 kB of memory. Processor Board ID FOC20291J6K Device name: cs2 bootflash: 53298520 kB Kernel uptime is 0 day(s), 0 hour(s), 3 minute(s), 42 second(s) Last reset at 277524 usecs after Mon Nov 2 22:45:12 2020 Reason: Reset due to upgrade System version: 10.4.5(M) Service: plugin Core Plugin, Ethernet Plugin Active Package(s):
-
Upgrade the EPLD image and reboot the switch.
show version module 1 epldinstall epld bootflash:<image_file_name> module allExample
cs2# show version module 1 epld EPLD Device Version --------------------------------------- MI FPGA 0x7 IO FPGA 0x17 MI FPGA2 0x2 GEM FPGA 0x2 GEM FPGA 0x2 GEM FPGA 0x2 GEM FPGA 0x2 cs2# install epld bootflash:n9000-epld.10.4.5.M.img module all Compatibility check: Module Type Upgradable Impact Reason ------ --------- ----------------- ----------- ----------- 1 SUP Yes disruptive Module Upgradable Retrieving EPLD versions.... Please wait. Images will be upgraded according to following table: Module Type EPLD Running-Version New-Version Upg-Required ------- ------ ----------------- ----------------- ------------ ------------ 1 SUP MI FPGA 0x07 0x07 No 1 SUP IO FPGA 0x17 0x19 Yes 1 SUP MI FPGA2 0x02 0x02 No The above modules require upgrade. The switch will be reloaded at the end of the upgrade Do you want to continue (y/n) ? [n] y Proceeding to upgrade Modules. Starting Module 1 EPLD Upgrade Module 1 : IO FPGA [Programming] : 100.00% ( 64 of 64 sectors) Module 1 EPLD upgrade is successful. Module Type Upgrade-Result -------- ----- -------------- 1 SUP Success EPLDs upgraded. Module 1 EPLD upgrade is successful. -
After the switch reboot, log in again and verify that the new version of EPLD loaded successfully.
show version module 1 epldExample
cs2# show version module 1 epld EPLD Device Version ---- -------------------------- ------- MI FPGA 0x7 IO FPGA 0x19 MI FPGA2 0x2 GEM FPGA 0x2 GEM FPGA 0x2 GEM FPGA 0x2 GEM FPGA 0x2
-
Copy the NX-OS software and EPLD image to the Nexus 9336C-FX2 switch.
copy <protocol>: bootflash: vrf managementExample
cs2# copy sftp: bootflash: vrf management Enter source filename: /code/nxos.10.5.3.F.bin Enter hostname for the sftp server: 172.19.2.1 Enter username: user1 Outbound-ReKey for 172.19.2.1:22 Inbound-ReKey for 172.19.2.1:22 user1@172.19.2.1's password: sftp> progress Progress meter enabled sftp> get /code/nxos.10.5.3.F.bin /bootflash/nxos.10.5.3.F.bin /code/nxos.10.5.3.F.bin 100% 1261MB 9.3MB/s 02:15 sftp> exit Copy complete, now saving to disk (please wait)... Copy complete. cs2# copy sftp: bootflash: vrf management Enter source filename: /code/n9000-epld.10.5.3.F.img Enter hostname for the sftp server: 172.19.2.1 Enter username: user1 Outbound-ReKey for 172.19.2.1:22 Inbound-ReKey for 172.19.2.1:22 user1@172.19.2.1's password: sftp> progress Progress meter enabled sftp> get /code/n9000-epld.10.5.3.F.img /bootflash/n9000-epld.10.5.3.F.img /code/n9000-epld.10.5.3.F.img 100% 161MB 9.5MB/s 00:16 sftp> exit Copy complete, now saving to disk (please wait)... Copy complete.
-
Verify the running version of the NX-OS software:
show versionExample
cs2# show version Cisco Nexus Operating System (NX-OS) Software TAC support: http://www.cisco.com/tac Copyright (C) 2002-2020, Cisco and/or its affiliates. All rights reserved. The copyrights to certain works contained in this software are owned by other third parties and used and distributed under their own licenses, such as open source. This software is provided "as is," and unless otherwise stated, there is no warranty, express or implied, including but not limited to warranties of merchantability and fitness for a particular purpose. Certain components of this software are licensed under the GNU General Public License (GPL) version 2.0 or GNU General Public License (GPL) version 3.0 or the GNU Lesser General Public License (LGPL) Version 2.1 or Lesser General Public License (LGPL) Version 2.0. A copy of each such license is available at http://www.opensource.org/licenses/gpl-2.0.php and http://opensource.org/licenses/gpl-3.0.html and http://www.opensource.org/licenses/lgpl-2.1.php and http://www.gnu.org/licenses/old-licenses/library.txt. Software BIOS: version 08.38 NXOS: version 10.5.3(F) BIOS compile time: 05/29/2020 NXOS image file is: bootflash:///nxos.10.5.3.F.bin NXOS compile time: 4/28/2020 21:00:00 [04/29/2020 02:28:31] Hardware cisco Nexus9000 C9336C-FX2 Chassis Intel(R) Xeon(R) CPU E5-2403 v2 @ 1.80GHz with 8154432 kB of memory. Processor Board ID FOC20291J6K Device name: cs2 bootflash: 53298520 kB Kernel uptime is 0 day(s), 0 hour(s), 3 minute(s), 42 second(s) Last reset at 157524 usecs after Mon Nov 2 18:32:06 2020 Reason: Reset Requested by CLI command reload System version: 9.3(4) Service: plugin Core Plugin, Ethernet Plugin Active Package(s): cs2#
-
Install the NX-OS and EPLD images.
The EPLD image is not automatically updated when you upgrade to NX-OS 10.5(3)(F) and later. After your upgrade completes and the switch reboots, you must run the same upgrade
install allcommand again.-
The second upgrade pass verifies, but skips the NX-OS upgrade; only the EPLD image is updated.
-
When you have already upgraded to NX-OS 10.5(3)(F) and later, both images are updated at the same time.
Installing the image file causes it to be loaded every time the switch is rebooted.
The following step shows an upgrade from NX-OS 10.4(5)(M) to 10.5(3)(F). The first pass performs the NX-OS upgrade and the second pass performs the EPLD image upgrade.
install all nxos bootflash:<nxos_image.bin>Example
cs2# install all nxos bootflash:nxos.10.5.3.F.bin <---- 1st pass updates NX-OS only Installer will perform compatibility check first. Please wait. Installer is forced disruptive Verifying image bootflash:/nxos.10.5.3.F.bin for boot variable "nxos". [] 100% -- SUCCESS Verifying image type. [] 100% -- SUCCESS Preparing "nxos" version info using image bootflash:/nxos.10.5.3.F.bin. [] 100% -- SUCCESS Preparing "bios" version info using image bootflash:/nxos.10.5.3.F.bin. [] 100% -- SUCCESS Performing module support checks. [] 100% -- SUCCESS Notifying services about system upgrade. [] 100% -- SUCCESS Compatibility check is done: Module Bootable Impact Install-type Reason ------ -------- --------------- ------------ --------- 1 yes Disruptive Reset Default upgrade is not hitless Images will be upgraded according to following table: Module Image Running-Version(pri:alt) New-Version Upg-Required ------- --------- ---------------------------------------- -------------------- ------------ 1 nxos 10.4(5)(M) 10.5(3)(F) yes 1 bios v05.47(04/28/2022):v05.47(04/28/2022) v05.53(01/22/2025) yes Switch will be reloaded for disruptive upgrade. Do you want to continue with the installation (y/n)? [n] y Install is in progress, please wait. Performing runtime checks. [] 100% -- SUCCESS Setting boot variables. [] 100% -- SUCCESS Performing configuration copy. [] 100% -- SUCCESS Module 1: Refreshing compact flash and upgrading bios/loader/bootrom. Warning: please do not remove or power off the module at this time. [] 100% -- SUCCESS [] 100% -- SUCCESS Finishing the upgrade, switch will reboot in 10 seconds. cs2# install all nxos bootflash:nxos.10.5.3.F.bin <---- 2nd pass updates EPLD image only Installer will perform compatibility check first. Please wait. Installer is forced disruptive Verifying image bootflash:/nxos.10.5.3.F.bin for boot variable "nxos". [] 100% -- SUCCESS Verifying EPLD/FPGA image //bootflash/nxos.10.5.3.F.bin. [] 100% -- SUCCESS Verifying image type. [] 100% -- SUCCESS Preparing "nxos" version info using image bootflash:/nxos.10.5.3.F.bin. [] 100% -- SUCCESS Preparing "bios" version info using image bootflash:/nxos.10.5.3.F.bin. [] 100% -- SUCCESS Performing module support checks. [] 100% -- SUCCESS Notifying services about system upgrade. [] 100% -- SUCCESS Compatibility check is done: Module bootable Impact Install-type Reason ------ -------- -------------- ------------ ------ 1 yes disruptive reset default upgrade is not hitless 27 yes disruptive none default upgrade is not hitless Images will be upgraded according to following table: Module Image Running-Version(pri:alt) New-Version Upg-Required -------- --------------------- ------------------------------------- ----------------- ------------ 27 nxos 10.4(5)(M) 10.5(3)(F) no 27 bios v05.53(01/22/2025):v05.47(04/28/2022) v05.53(01/22/2025) no FPGA microcode will be upgraded according to following table: Module Type EPLD Running-Version Flashed-Version* New-Version Upg-Required ------ ---- ----------------- --------------- ---------------- ----------- ------------ 27 SUP MI FPGA 0x5 0x5 0x5 No 27 SUP IO FPGA 0x17 0x17 0x18 Yes If Running-Version and Flashed-Version are different it implies that the system has not yet been reloaded for the new version to take effect. EPLD Upgrade may result in multiple modules going offline. Switch will be reloaded for disruptive upgrade. Do you want to continue with the installation (y/n)? [n] y Install is in progress, please wait. Setting boot variables. [] 100% -- SUCCESS Performing configuration copy. [] 100% -- SUCCESS Performing configuration copy. [] 100% -- SUCCESS Module 1: Refreshing compact flash and upgrading bios/loader/bootrom. Warning: please do not remove or power off the module at this time. Module 1: Refreshing compact flash and upgrading bios/loader/bootrom. Warning: please do not remove or power off the module at this time. [] 100% -- SUCCESS Module 27: Refreshing compact flash and upgrading bios/loader/bootrom. Warning: please do not remove or power off the module at this time. Module 27: Refreshing compact flash and upgrading bios/loader/bootrom. Warning: please do not remove or power off the module at this time. [] 100% -- SUCCESS EPLD/FPGA upgrade can take up to 4 mins. Performing EPLD/FPGA upgrade. [] 100% -- SUCCESS Install has been successful.
-
-
Verify the new version of NX-OS software after the switch has rebooted:
show versionExample
cs2# show version Cisco Nexus Operating System (NX-OS) Software TAC support: http://www.cisco.com/tac Copyright (C) 2002-2020, Cisco and/or its affiliates. All rights reserved. The copyrights to certain works contained in this software are owned by other third parties and used and distributed under their own licenses, such as open source. This software is provided "as is," and unless otherwise stated, there is no warranty, express or implied, including but not limited to warranties of merchantability and fitness for a particular purpose. Certain components of this software are licensed under the GNU General Public License (GPL) version 2.0 or GNU General Public License (GPL) version 3.0 or the GNU Lesser General Public License (LGPL) Version 2.1 or Lesser General Public License (LGPL) Version 2.0. A copy of each such license is available at http://www.opensource.org/licenses/gpl-2.0.php and http://opensource.org/licenses/gpl-3.0.html and http://www.opensource.org/licenses/lgpl-2.1.php and http://www.gnu.org/licenses/old-licenses/library.txt. Software BIOS: version 05.33 NXOS: version 10.5.3(f) BIOS compile time: 09/08/2018 NXOS image file is: bootflash:///nxos.10.5.3.F.bin NXOS compile time: 11/4/2018 21:00:00 [11/05/2018 06:11:06] Hardware cisco Nexus9000 C9336C-FX2 Chassis Intel(R) Xeon(R) CPU E5-2403 v2 @ 1.80GHz with 8154432 kB of memory. Processor Board ID FOC20291J6K Device name: cs2 bootflash: 53298520 kB Kernel uptime is 0 day(s), 0 hour(s), 3 minute(s), 42 second(s) Last reset at 277524 usecs after Mon Nov 2 22:45:12 2020 Reason: Reset due to upgrade System version: 9.3(4) Service: plugin Core Plugin, Ethernet Plugin Active Package(s):
-
After the switch reboot, log in again and verify that the new version of EPLD loaded successfully.
show version module 1 epldExample
cs2# show version module 1 epld EPLD Device Version --------------------------------------- MI FPGA 0x7 IO FPGA 0x19 MI FPGA2 0x2 GEM FPGA 0x2 GEM FPGA 0x2 GEM FPGA 0x2 GEM FPGA 0x2
-
Copy the NX-OS and EPLD image to the first switch.
copy <protocol>: bootflash: vrf managementThe example uses sftp.
Example
cs2# copy sftp: bootflash: vrf management Enter source filename: nxos64-cs.10.5.4.M.bin Enter hostname for the sftp server: 172.19.2.1 Enter username: user1 Outbound-ReKey for 172.19.2.1:22 Inbound-ReKey for 172.19.2.1:22 user1@172.19.2.1's password: sftp> progress Progress meter enabled sftp> get /code/nxos64-cs.10.5.4.M.bin /bootflash/nxos64-cs.10.5.4.M.bin /code/nxos64-cs.10.5.4.M.bin 100% 1261MB 9.3MB/s 02:15 sftp> exit Copy complete, now saving to disk (please wait)... Copy complete.
-
Verify the running version of NX-OS:
show versionExample
cs2# show version Cisco Nexus Operating System (NX-OS) Software TAC support: http://www.cisco.com/tac Copyright (C) 2002-2025, Cisco and/or its affiliates. All rights reserved. The copyrights to certain works contained in this software are owned by other third parties and used and distributed under their own licenses, such as open source. This software is provided "as is," and unless otherwise stated, there is no warranty, express or implied, including but not limited to warranties of merchantability and fitness for a particular purpose. Certain components of this software are licensed under the GNU General Public License (GPL) version 2.0 or GNU General Public License (GPL) version 3.0 or the GNU Lesser General Public License (LGPL) Version 2.1 or Lesser General Public License (LGPL) Version 2.0. A copy of each such license is available at http://www.opensource.org/licenses/gpl-2.0.php and http://opensource.org/licenses/gpl-3.0.html and http://www.opensource.org/licenses/lgpl-2.1.php and http://www.gnu.org/licenses/old-licenses/library.txt. Software BIOS: version 01.14 NXOS: version 10.5(3)(F) [Feature Release] Host NXOS: version 10.5(3)(F) BIOS compile time: 11/25/2024 NXOS image file is: bootflash:///nxos.10.5.3.F.bin NXOS compile time: 02/27/2026 12:00:00 [03/14/2026 05:25:50] NXOS boot mode: LXC Hardware cisco Nexus9000 C9336C-FX2 Chassis Intel(R) Xeon(R) CPU D-1633N @ 2.50GHz with 32802156 kB of memory. Processor Board ID FLMXXXXXXXX Device name: cs2 bootflash: 115802886 kB Kernel uptime is 5 day(s), 2 hour(s), 13 minute(s), 21 second(s) Last reset at 3580 usecs after Thu Mar 5 15:55:08 2026 Reason: Reset Requested by CLI command reload System version: 10.5.3(f) Service: plugin Core Plugin, Ethernet Plugin Active Package(s): cs2#
-
Install the NX-OS and EPLD image.
Installing the image file causes it to be loaded every time the switch is rebooted.
install all nxos bootflash:<nxos_image.bin>The example uses NX-OS version 10.5(4)(M).
Example
cs2# install all nxos bootflash:nxos64-cs.10.5.4.M.bin Installer will perform compatibility check first. Please wait. Installer is forced disruptive Verifying image bootflash:/nxos64-cs.10.5.4.M.bin for boot variable "nxos". [] 100% -- SUCCESS Verifying image type. [] 100% -- SUCCESS Preparing "nxos" version info using image bootflash:/nxos64-cs.10.5.4.M.bin. [] 100% -- SUCCESS Preparing "bios" version info using image bootflash:/nxos64-cs.10.5.4.M.bin. [] 100% -- SUCCESS Performing module support checks. [] 100% -- SUCCESS Notifying services about system upgrade. [] 100% -- SUCCESS Compatibility check is done: Module Bootable Impact Install-type Reason ------ -------- --------------- ------------ --------- 1 yes Disruptive Reset Default upgrade is not hitless Images will be upgraded according to following table: Module Image Running-Version(pri:alt) New-Version Upg-Required ------- --------- ----------------------------------- ------------------- ------------ 1 nxos 10.5(3)(F) 10.5(4)(M) Yes 1 bios <current_version> <new_version> No Switch will be reloaded for disruptive upgrade. Do you want to continue with the installation (y/n)? [n] y Install is in progress, please wait. Performing runtime checks. [] 100% -- SUCCESS Setting boot variables. [] 100% -- SUCCESS Performing configuration copy. [] 100% -- SUCCESS Module 1: Refreshing compact flash and upgrading bios/loader/bootrom. Warning: please do not remove or power off the module at this time. [] 100% -- SUCCESS Finishing the upgrade, switch will reboot in 10 seconds.
-
Verify the new version of NX-OS after the switch has rebooted:
show versionExample
cs2# show version Cisco Nexus Operating System (NX-OS) Software TAC support: http://www.cisco.com/tac Copyright (C) 2002-2025, Cisco and/or its affiliates. All rights reserved. The copyrights to certain works contained in this software are owned by other third parties and used and distributed under their own licenses, such as open source. This software is provided "as is," and unless otherwise stated, there is no warranty, express or implied, including but not limited to warranties of merchantability and fitness for a particular purpose. Certain components of this software are licensed under the GNU General Public License (GPL) version 2.0 or GNU General Public License (GPL) version 3.0 or the GNU Lesser General Public License (LGPL) Version 2.1 or Lesser General Public License (LGPL) Version 2.0. A copy of each such license is available at http://www.opensource.org/licenses/gpl-2.0.php and http://opensource.org/licenses/gpl-3.0.html and http://www.opensource.org/licenses/lgpl-2.1.php and http://www.gnu.org/licenses/old-licenses/library.txt. Software BIOS: version 01.14 NXOS: version 10.5(4)(M) [Feature Release] Host NXOS: version 10.5(4)(M) BIOS compile time: 11/25/2024 NXOS image file is: bootflash:///nxos64-cs.10.5.4.M.bin NXOS compile time: 05/30/2026 12:00:00 [06/14/2026 05:25:50] NXOS boot mode: LXC Hardware cisco Nexus9000 C9336C-FX2 Chassis Intel(R) Xeon(R) CPU D-1633N @ 2.50GHz with 32802156 kB of memory. Processor Board ID FLMXXXXXXXX Device name: cs2 bootflash: 115802886 kB Kernel uptime is 5 day(s), 2 hour(s), 13 minute(s), 21 second(s) Last reset at 3580 usecs after Thu Jun 5 15:55:08 2026 Reason: Reset Requested by CLI command reload System version: 10.5(4)(M) Service: plugin Core Plugin, Ethernet Plugin Active Package(s):
-
Verify that the new version of EPLD loaded successfully:
show install epld statusThe example uses EPLD version 10.5(4)(M).
Example
cs2# show install epld status 1) Module 2 upgraded on Mon May 4 09:17:08 2026 (631239 us) EPLD Install Image: EPLD image file 10.5.4.M. built on Fri Apr 24 19:49:50 2026 Status: EPLD Upgrade was Successful EPLD Curr Ver Old Ver ------------------------------------------------------- MI FPGA 0x10014 0x1000b IO FPGA 0x10033 0x10006 2) Module 27 upgraded on Wed Apr 1 15:03:13 2026 (193093 us) EPLD Install Image: EPLD image file 10.5.4.M. built on Mon Mar 16 03:28:47 2026 Status: EPLD Upgrade was Successful EPLD Curr Ver Old Ver ------------------------------------------------------- IO FPGA 0x1002d 0x10020 3) Module 28 upgraded on Wed Apr 1 14:59:53 2026 (843907 us) EPLD Install Image: EPLD image file 10.5.4.M. built on Mon Mar 16 03:28:47 2026 Status: EPLD Upgrade was Successful EPLD Curr Ver Old Ver ------------------------------------------------------- IO FPGA 0x1002d 0x10020 4) Module 27 upgraded on Fri Mar 27 21:55:55 2026 (132290 us) EPLD Install Image: EPLD image file 10.5.4.M. built on Fri Dec 12 11:15:18 2025 Status: EPLD Upgrade was Successful EPLD Curr Ver Old Ver ------------------------------------------------------- FM-19 FPGA 0x10006 0x10000 FM-20 FPGA 0x10006 0x10000 FM-21 FPGA 0x10006 0x10000 FM-22 FPGA 0x10006 0x10000 FM-23 FPGA 0x10006 0x10000 FM-24 FPGA 0x10006 0x10000 FM-25 FPGA 0x10006 0x10000 FM-26 FPGA 0x10006 0x10000 5) Module 27 upgraded on Fri Mar 27 21:50:16 2026 ( 4899 us) EPLD Install Image: EPLD image file 10.5.4.M. built on Fri Dec 12 11:15:18 2025 Status: EPLD Upgrade was Successful EPLD Curr Ver Old Ver ------------------------------------------------------- FM-19 FPGA 0x10006 0x10000 FM-20 FPGA 0x10006 0x10000 FM-21 FPGA 0x10006 0x10000 FM-22 FPGA 0x10006 0x10000 FM-23 FPGA 0x10006 0x10000 FM-24 FPGA 0x10006 0x10000 FM-25 FPGA 0x10006 0x10000 FM-26 FPGA 0x10006 0x10000 6) Module 27 upgraded on Fri Mar 27 21:44:36 2026 (775397 us) EPLD Install Image: EPLD image file 10.5.4.M. built on Fri Dec 12 11:15:18 2025 Status: EPLD Upgrade was Successful EPLD Curr Ver Old Ver ------------------------------------------------------- FM-19 FPGA 0x10006 0x10000 FM-20 FPGA 0x10006 0x10000 FM-21 FPGA 0x10006 0x10000 FM-22 FPGA 0x10006 0x10000 FM-23 FPGA 0x10006 0x10000 FM-24 FPGA 0x10006 0x10000 FM-25 FPGA 0x10006 0x10000 FM-26 FPGA 0x10006 0x10000 7) Module 27 upgraded on Fri Mar 27 21:38:58 2026 (330582 us) EPLD Install Image: EPLD image file 10.5.4.M. built on Fri Dec 12 11:15:18 2025 Status: EPLD Upgrade was Successful EPLD Curr Ver Old Ver ------------------------------------------------------- FM-19 FPGA 0x10006 0x10000 FM-20 FPGA 0x10006 0x10000 FM-21 FPGA 0x10006 0x10000 FM-22 FPGA 0x10006 0x10000 FM-23 FPGA 0x10006 0x10000 FM-24 FPGA 0x10006 0x10000 FM-25 FPGA 0x10006 0x10000 FM-26 FPGA 0x10006 0x10000 8) Module 27 upgraded on Fri Mar 27 21:33:19 2026 (112679 us) EPLD Install Image: EPLD image file 10.5.4.M. built on Fri Dec 12 11:15:18 2025 Status: EPLD Upgrade was Successful EPLD Curr Ver Old Ver ------------------------------------------------------- FM-19 FPGA 0x10006 0x10000 FM-20 FPGA 0x10006 0x10000 FM-21 FPGA 0x10006 0x10000 FM-22 FPGA 0x10006 0x10000 FM-23 FPGA 0x10006 0x10000 FM-24 FPGA 0x10006 0x10000 FM-25 FPGA 0x10006 0x10000 FM-26 FPGA 0x10006 0x10000 9) Module 27 upgraded on Fri Mar 27 21:27:41 2026 (957182 us) EPLD Install Image: EPLD image file 10.5.4.M. built on Fri Dec 12 11:15:18 2025 Status: EPLD Upgrade was Successful EPLD Curr Ver Old Ver ------------------------------------------------------- FM-19 FPGA 0x10006 0x10000 FM-20 FPGA 0x10006 0x10000 FM-21 FPGA 0x10006 0x10000 FM-22 FPGA 0x10006 0x10000 FM-23 FPGA 0x10006 0x10000 FM-24 FPGA 0x10006 0x10000 FM-25 FPGA 0x10006 0x10000 FM-26 FPGA 0x10006 0x10000 10) Module 27 upgraded on Fri Mar 27 21:22:03 2026 ( 90303 us) EPLD Install Image: EPLD image file 10.5.4.M. built on Fri Dec 12 11:15:18 2025 Status: EPLD Upgrade was Successful EPLD Curr Ver Old Ver ------------------------------------------------------- FM-19 FPGA 0x10006 0x10000 FM-20 FPGA 0x10006 0x10000 FM-21 FPGA 0x10006 0x10000 FM-22 FPGA 0x10006 0x10000 FM-23 FPGA 0x10006 0x10000 FM-24 FPGA 0x10006 0x10000 FM-25 FPGA 0x10006 0x10000 FM-26 FPGA 0x10006 0x10000 11) Module 1 upgraded on Fri Mar 27 21:16:24 2026 (240369 us) EPLD Install Image: EPLD image file 10.5.4.M. built on Fri Dec 12 11:15:18 2025 Status: EPLD Upgrade was Successful EPLD Curr Ver Old Ver ------------------------------------------------------- MI FPGA 0x10014 0x1000b IO FPGA 0x10033 0x10006
-
Verify the health of cluster ports on the cluster.
-
Verify that cluster ports are up and healthy across all nodes in the cluster:
network port show -ipspace ClusterShow example
cluster1::*> network port show -ipspace Cluster Node: cluster1-01 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ------------ -------- ------ e0a Cluster Cluster up 9000 auto/100000 healthy false e0b Cluster Cluster up 9000 auto/100000 healthy false Node: cluster1-02 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ------------ -------- ------ e0a Cluster Cluster up 9000 auto/100000 healthy false e0b Cluster Cluster up 9000 auto/100000 healthy false Node: cluster1-03 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ------------ -------- ------ e0a Cluster Cluster up 9000 auto/100000 healthy false e0b Cluster Cluster up 9000 auto/100000 healthy false Node: cluster1-04 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ----------- -------- ------ e0a Cluster Cluster up 9000 auto/100000 healthy false e0b Cluster Cluster up 9000 auto/100000 healthy false -
Verify the switch health from the cluster.
network device-discovery show -protocol cdpShow example
cluster1::*> network device-discovery show -protocol cdp Node/ Local Discovered Protocol Port Device (LLDP: ChassisID) Interface Platform ----------- ------ ------------------------- ----------------- -------------- cluster1-01/cdp e0a cs1 Ethernet1/7 N9K-C9336C-FX2 e0b cs2 Ethernet1/7 N9K-C9336C-FX2 cluster01-2/cdp e0a cs1 Ethernet1/8 N9K-C9336C-FX2 e0b cs2 Ethernet1/8 N9K-C9336C-FX2 cluster01-3/cdp e0a cs1 Ethernet1/1/1 N9K-C9336C-FX2 e0b cs2 Ethernet1/1/1 N9K-C9336C-FX2 cluster1-04/cdp e0a cs1 Ethernet1/1/2 N9K-C9336C-FX2 e0b cs2 Ethernet1/1/2 N9K-C9336C-FX2 cluster1::*> system cluster-switch show -is-monitoring-enabled-operational true Switch Type Address Model --------------------------- ------------------ ---------------- -------------- cs1 cluster-network 10.233.205.90 N9K-C9336C-FX2 Serial Number: FOCXXXXXXGD Is Monitored: true Reason: None Software Version: Cisco Nexus Operating System (NX-OS) Software, Version 9.3(5) Version Source: CDP cs2 cluster-network 10.233.205.91 N9K-C9336C-FX2 Serial Number: FOCXXXXXXGS Is Monitored: true Reason: None Software Version: Cisco Nexus Operating System (NX-OS) Software, Version 9.3(5) Version Source: CDP 2 entries were displayed.You might observe the following output on the cs1 switch console depending on the RCF version previously loaded on the switch:
2020 Nov 17 16:07:18 cs1 %$ VDC-1 %$ %STP-2-UNBLOCK_CONSIST_PORT: Unblocking port port-channel1 on VLAN0092. Port consistency restored. 2020 Nov 17 16:07:23 cs1 %$ VDC-1 %$ %STP-2-BLOCK_PVID_PEER: Blocking port-channel1 on VLAN0001. Inconsistent peer vlan. 2020 Nov 17 16:07:23 cs1 %$ VDC-1 %$ %STP-2-BLOCK_PVID_LOCAL: Blocking port-channel1 on VLAN0092. Inconsistent local vlan.
-
-
Verify that the cluster is healthy:
cluster showShow example
cluster1::*> cluster show Node Health Eligibility Epsilon -------------------- -------- ------------- ------- cluster1-01 true true false cluster1-02 true true false cluster1-03 true true true cluster1-04 true true false 4 entries were displayed. cluster1::*>
-
Repeat steps 6 to 8 to install the NX-OS software on switch cs1.
-
Verify the connectivity of the remote cluster interfaces before enabling auto-revert on the cluster LIFs:
You can use the network interface check cluster-connectivity command to start an accessibility check for cluster connectivity and then display the details:
network interface check cluster-connectivity start and network interface check cluster-connectivity show
cluster1::*> network interface check cluster-connectivity start
NOTE: Wait for a number of seconds before running the show command to display the details.
cluster1::*> network interface check cluster-connectivity show
Source Destination Packet
Node Date LIF LIF Loss
------ -------------------------- -------------------- ------------------- -----------
cluster1-01
3/5/2022 19:21:18 -06:00 cluster1-01_clus2 cluster1-02-clus1 none
3/5/2022 19:21:20 -06:00 cluster1-01_clus2 cluster1-02_clus2 none
cluster1-02
3/5/2022 19:21:18 -06:00 cluster1-02_clus2 cluster1-01_clus1 none
3/5/2022 19:21:20 -06:00 cluster1-02_clus2 cluster1-01_clus2 none
For all ONTAP releases, you can also use the cluster ping-cluster -node <name> command to check the connectivity:
cluster ping-cluster -node <name>
cluster1::*> cluster ping-cluster -node local
Host is cluster1-02
Getting addresses from network interface table...
Cluster cluster1-01_clus1 169.254.209.69 cluster1-01 e0a
Cluster cluster1-01_clus2 169.254.49.125 cluster1-01 e0b
Cluster cluster1-02_clus1 169.254.47.194 cluster1-02 e0a
Cluster cluster1-02_clus2 169.254.19.183 cluster1-02 e0b
Local = 169.254.47.194 169.254.19.183
Remote = 169.254.209.69 169.254.49.125
Cluster Vserver Id = 4294967293
Ping status:
Basic connectivity succeeds on 4 path(s)
Basic connectivity fails on 0 path(s)
Detected 9000 byte MTU on 4 path(s):
Local 169.254.19.183 to Remote 169.254.209.69
Local 169.254.19.183 to Remote 169.254.49.125
Local 169.254.47.194 to Remote 169.254.209.69
Local 169.254.47.194 to Remote 169.254.49.125
Larger than PMTU communication succeeds on 4 path(s)
RPC status:
2 paths up, 0 paths down (tcp check)
2 paths up, 0 paths down (udp check)
-
Enable auto-revert on the cluster LIFs.
network interface modify -vserver Cluster -lif * -auto-revert true -
Verify that the cluster LIFs have reverted to their home port:
network interface show -vserver ClusterShow example
cluster1::*> network interface show -vserver Cluster Logical Status Network Current Current Is Vserver Interface Admin/Oper Address/Mask Node Port Home ----------- ------------------ ---------- ------------------ ------------------- ------- ---- Cluster cluster1-01_clus1 up/up 169.254.3.4/23 cluster1-01 e0b true cluster1-01_clus2 up/up 169.254.3.5/23 cluster1-01 e0b true cluster1-02_clus1 up/up 169.254.3.8/23 cluster1-02 e0b true cluster1-02_clus2 up/up 169.254.3.9/23 cluster1-02 e0b true cluster1-03_clus1 up/up 169.254.1.3/23 cluster1-03 e0b true cluster1-03_clus2 up/up 169.254.1.1/23 cluster1-03 e0b true cluster1-04_clus1 up/up 169.254.1.6/23 cluster1-04 e0b true cluster1-04_clus2 up/up 169.254.1.7/23 cluster1-04 e0b true cluster1::*>If cluster LIFs have not returned to their home ports, manually revert them from the local node:
network interface revert -vserver Cluster -lif <lif_name>
After you've installed or upgraded the NX-OS software, you can install or upgrade the Reference Configuration File (RCF).