Install the NX-OS software
Contributors
Suggest changes
PDFs
You can use this procedure to install the NX-OS software on the Nexus 3232C cluster switch.
Review requirements
What you'll need
-
A current backup of the switch configuration.
-
A fully functioning cluster (no errors in the logs or similar issues).
-
Cisco Ethernet switch page. Consult the switch compatibility table for the supported ONTAP and NX-OS versions.
-
Cisco Nexus 3000 Series Switches. Refer to the appropriate software and upgrade guides available on the Cisco web site for complete documentation on the Cisco switch upgrade and downgrade procedures.
Install the software
The procedure requires the use of both ONTAP commands and Cisco Nexus 3000 Series Switches commands; ONTAP commands are used unless otherwise indicated.
Be sure to complete the procedure in Prepare to install NX-OS and RCF, and then follow the steps below.
Steps
-
Connect the cluster switch to the management network.
-
Use the
pingcommand to verify connectivity to the server hosting the NX-OS software and the RCF.Show example
This example verifies that the switch can reach the server at IP address 172.19.2.1:
cs2# ping 172.19.2.1 Pinging 172.19.2.1 with 0 bytes of data: Reply From 172.19.2.1: icmp_seq = 0. time= 5910 usec.
-
Display the cluster ports on each node that are connected to the cluster switches:
network device-discovery showShow example
cluster1::*> network device-discovery show Node/ Local Discovered Protocol Port Device (LLDP: ChassisID) Interface Platform ----------- ------ ------------------------- ---------------- ---------- cluster1-01/cdp e0a cs1 Ethernet1/7 N3K-C3232C e0d cs2 Ethernet1/7 N3K-C3232C cluster1-02/cdp e0a cs1 Ethernet1/8 N3K-C3232C e0d cs2 Ethernet1/8 N3K-C3232C cluster1-03/cdp e0a cs1 Ethernet1/1/1 N3K-C3232C e0b cs2 Ethernet1/1/1 N3K-C3232C cluster1-04/cdp e0a cs1 Ethernet1/1/2 N3K-C3232C e0b cs2 Ethernet1/1/2 N3K-C3232C cluster1::*> -
Check the administrative and operational status of each cluster port.
-
Verify that all the cluster ports are up with a healthy status:
network port show -role clusterShow example
cluster1::*> network port show -role cluster Node: cluster1-01 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ----------- -------- ------ e0a Cluster Cluster up 9000 auto/100000 healthy false e0d Cluster Cluster up 9000 auto/100000 healthy false Node: cluster1-02 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ----------- -------- ------ e0a Cluster Cluster up 9000 auto/100000 healthy false e0d Cluster Cluster up 9000 auto/100000 healthy false 8 entries were displayed. Node: cluster1-03 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ----------- -------- ------ e0a Cluster Cluster up 9000 auto/10000 healthy false e0b Cluster Cluster up 9000 auto/10000 healthy false Node: cluster1-04 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ----------- -------- ------ e0a Cluster Cluster up 9000 auto/10000 healthy false e0b Cluster Cluster up 9000 auto/10000 healthy false cluster1::*> -
Verify that all the cluster interfaces (LIFs) are on the home port:
network interface show -role clusterShow example
cluster1::*> network interface show -role cluster Logical Status Network Current Current Is Vserver Interface Admin/Oper Address/Mask Node Port Home ----------- ------------------ ---------- ----------------- ------------ ------- ---- Cluster cluster1-01_clus1 up/up 169.254.3.4/23 cluster1-01 e0a true cluster1-01_clus2 up/up 169.254.3.5/23 cluster1-01 e0d true cluster1-02_clus1 up/up 169.254.3.8/23 cluster1-02 e0a true cluster1-02_clus2 up/up 169.254.3.9/23 cluster1-02 e0d true cluster1-03_clus1 up/up 169.254.1.3/23 cluster1-03 e0a true cluster1-03_clus2 up/up 169.254.1.1/23 cluster1-03 e0b true cluster1-04_clus1 up/up 169.254.1.6/23 cluster1-04 e0a true cluster1-04_clus2 up/up 169.254.1.7/23 cluster1-04 e0b true 8 entries were displayed. cluster1::*> -
Verify that the cluster displays information for both cluster switches:
system cluster-switch show -is-monitoring-enabled-operational trueShow example
cluster1::*> system cluster-switch show -is-monitoring-enabled-operational true Switch Type Address Model --------------------------- ------------------ ---------------- ---------- cs1 cluster-network 10.233.205.90 N3K-C3232C Serial Number: FOCXXXXXXGD Is Monitored: true Reason: None Software Version: Cisco Nexus Operating System (NX-OS) Software, Version 9.3(5) Version Source: CDP cs2 cluster-network 10.233.205.91 N3K-C3232C Serial Number: FOCXXXXXXGS Is Monitored: true Reason: None Software Version: Cisco Nexus Operating System (NX-OS) Software, Version 9.3(5) Version Source: CDP cluster1::*>
-
-
Disable auto-revert on the cluster LIFs. The cluster LIFs fail over to the partner cluster switch and remain there as you perform the upgrade procedure on the targeted switch:
network interface modify -vserver Cluster -lif * -auto-revert false -
Copy the NX-OS software and EPLD images to the Nexus 3232C switch.
Show example
cs2# copy sftp: bootflash: vrf management Enter source filename: /code/nxos.9.3.4.bin Enter hostname for the sftp server: 172.19.2.1 Enter username: user1 Outbound-ReKey for 172.19.2.1:22 Inbound-ReKey for 172.19.2.1:22 user1@172.19.2.1's password: sftp> progress Progress meter enabled sftp> get /code/nxos.9.3.4.bin /bootflash/nxos.9.3.4.bin /code/nxos.9.3.4.bin 100% 1261MB 9.3MB/s 02:15 sftp> exit Copy complete, now saving to disk (please wait)... Copy complete. cs2# copy sftp: bootflash: vrf management Enter source filename: /code/n9000-epld.9.3.4.img Enter hostname for the sftp server: 172.19.2.1 Enter username: user1 Outbound-ReKey for 172.19.2.1:22 Inbound-ReKey for 172.19.2.1:22 user1@172.19.2.1's password: sftp> progress Progress meter enabled sftp> get /code/n9000-epld.9.3.4.img /bootflash/n9000-epld.9.3.4.img /code/n9000-epld.9.3.4.img 100% 161MB 9.5MB/s 00:16 sftp> exit Copy complete, now saving to disk (please wait)... Copy complete.
-
Verify the running version of the NX-OS software:
show versionShow example
cs2# show version Cisco Nexus Operating System (NX-OS) Software TAC support: http://www.cisco.com/tac Copyright (C) 2002-2019, Cisco and/or its affiliates. All rights reserved. The copyrights to certain works contained in this software are owned by other third parties and used and distributed under their own licenses, such as open source. This software is provided "as is," and unless otherwise stated, there is no warranty, express or implied, including but not limited to warranties of merchantability and fitness for a particular purpose. Certain components of this software are licensed under the GNU General Public License (GPL) version 2.0 or GNU General Public License (GPL) version 3.0 or the GNU Lesser General Public License (LGPL) Version 2.1 or Lesser General Public License (LGPL) Version 2.0. A copy of each such license is available at http://www.opensource.org/licenses/gpl-2.0.php and http://opensource.org/licenses/gpl-3.0.html and http://www.opensource.org/licenses/lgpl-2.1.php and http://www.gnu.org/licenses/old-licenses/library.txt. Software BIOS: version 08.37 NXOS: version 9.3(3) BIOS compile time: 01/28/2020 NXOS image file is: bootflash:///nxos.9.3.3.bin NXOS compile time: 12/22/2019 2:00:00 [12/22/2019 14:00:37] Hardware cisco Nexus3000 C3232C Chassis (Nexus 9000 Series) Intel(R) Xeon(R) CPU E5-2403 v2 @ 1.80GHz with 8154432 kB of memory. Processor Board ID FOCXXXXXXGD Device name: cs2 bootflash: 53298520 kB Kernel uptime is 0 day(s), 0 hour(s), 3 minute(s), 36 second(s) Last reset at 74117 usecs after Tue Nov 24 06:24:23 2020 Reason: Reset Requested by CLI command reload System version: 9.3(3) Service: plugin Core Plugin, Ethernet Plugin Active Package(s): cs2#
-
Install the NX-OS image.
Installing the image file causes it to be loaded every time the switch is rebooted.
Show example
cs2# install all nxos bootflash:nxos.9.3.4.bin Installer will perform compatibility check first. Please wait. Installer is forced disruptive Verifying image bootflash:/nxos.9.3.4.bin for boot variable "nxos". [] 100% -- SUCCESS Verifying image type. [] 100% -- SUCCESS Preparing "nxos" version info using image bootflash:/nxos.9.3.4.bin. [] 100% -- SUCCESS Preparing "bios" version info using image bootflash:/nxos.9.3.4.bin. [] 100% -- SUCCESS Performing module support checks. [] 100% -- SUCCESS Notifying services about system upgrade. [] 100% -- SUCCESS Compatibility check is done: Module bootable Impact Install-type Reason ------- ----------------- ------------------- ------------- ---------- 1 Yes Disruptive Reset Default upgrade is not hitless Images will be upgraded according to following table: Module Image Running-Version(pri:alt) New-Version Upg-Required ------------ ----------- --------------------------------------- -------------------- ------------ 1 nxos 9.3(3) 9.3(4) yes 1 bios v08.37(01/28/2020):v08.32(10/18/2016) v08.37(01/28/2020) no Switch will be reloaded for disruptive upgrade. Do you want to continue with the installation (y/n)? [n] y Install is in progress, please wait. Performing runtime checks. [] 100% -- SUCCESS Setting boot variables. [] 100% -- SUCCESS Performing configuration copy. [] 100% -- SUCCESS Module 1: Refreshing compact flash and upgrading bios/loader/bootrom. Warning: please do not remove or power off the module at this time. [] 100% -- SUCCESS Finishing the upgrade, switch will reboot in 10 seconds. cs2# -
Verify the new version of NX-OS software after the switch has rebooted:
show versionShow example
cs2# show version Cisco Nexus Operating System (NX-OS) Software TAC support: http://www.cisco.com/tac Copyright (C) 2002-2020, Cisco and/or its affiliates. All rights reserved. The copyrights to certain works contained in this software are owned by other third parties and used and distributed under their own licenses, such as open source. This software is provided "as is," and unless otherwise stated, there is no warranty, express or implied, including but not limited to warranties of merchantability and fitness for a particular purpose. Certain components of this software are licensed under the GNU General Public License (GPL) version 2.0 or GNU General Public License (GPL) version 3.0 or the GNU Lesser General Public License (LGPL) Version 2.1 or Lesser General Public License (LGPL) Version 2.0. A copy of each such license is available at http://www.opensource.org/licenses/gpl-2.0.php and http://opensource.org/licenses/gpl-3.0.html and http://www.opensource.org/licenses/lgpl-2.1.php and http://www.gnu.org/licenses/old-licenses/library.txt. Software BIOS: version 08.37 NXOS: version 9.3(4) BIOS compile time: 01/28/2020 NXOS image file is: bootflash:///nxos.9.3.4.bin NXOS compile time: 4/28/2020 21:00:00 [04/29/2020 06:28:31] Hardware cisco Nexus3000 C3232C Chassis (Nexus 9000 Series) Intel(R) Xeon(R) CPU E5-2403 v2 @ 1.80GHz with 8154432 kB of memory. Processor Board ID FOCXXXXXXGS Device name: rtpnpi-mcc01-8200-ms-A1 bootflash: 53298520 kB Kernel uptime is 0 day(s), 0 hour(s), 3 minute(s), 14 second(s) Last reset at 196755 usecs after Tue Nov 24 06:37:36 2020 Reason: Reset due to upgrade System version: 9.3(3) Service: plugin Core Plugin, Ethernet Plugin Active Package(s): cs2#
-
Upgrade the EPLD image and reboot the switch.
Show example
cs2# show version module 1 epld EPLD Device Version --------------------------------------- MI FPGA 0x12 IO FPGA 0x11 cs2# install epld bootflash:n9000-epld.9.3.4.img module 1 Compatibility check: Module Type Upgradable Impact Reason ------ ----------------- ---------- ---------- ----------------- 1 SUP Yes Disruptive Module Upgradable Retrieving EPLD versions.... Please wait. Images will be upgraded according to following table: Module Type EPLD Running-Version New-Version Upg-Required ------ ---- ----------------- --------------- ----------- ------------ 1 SUP MI FPGA 0x12 0x12 No 1 SUP IO FPGA 0x11 0x12 Yes The above modules require upgrade. The switch will be reloaded at the end of the upgrade Do you want to continue (y/n) ? [n] y Proceeding to upgrade Modules. Starting Module 1 EPLD Upgrade Module 1 : IO FPGA [Programming] : 100.00% ( 64 of 64 sectors) Module 1 EPLD upgrade is successful. Module Type Upgrade-Result ------ ------------------ -------------- 1 SUP Success Module 1 EPLD upgrade is successful. cs2# -
If you are upgrading to NX-OS version 9.3(11), you must upgrade the EPLD
goldenimage and reboot the switch once again. Otherwise, skip to step 12.See EPLD Upgrade Release Notes, Release 9.3(11) for further details.
Show example
cs2# install epld bootflash:n9000-epld.9.3.11.img module 1 golden Digital signature verification is successful Compatibility check: Module Type Upgradable Impact Reason ------ ----------------- ------------- ---------- ----------------- 1 SUP Yes Disruptive Module Upgradable Retrieving EPLD versions.... Please wait. The above modules require upgrade. The switch will be reloaded at the end of the upgrade Do you want to continue (y/n) ? [n] y Proceeding to upgrade Modules. Starting Module 1 EPLD Upgrade Module 1 : MI FPGA [Programming] : 100.00% ( 64 of 64 sect) Module 1 : IO FPGA [Programming] : 100.00% ( 64 of 64 sect) Module 1 EPLD upgrade is successful. Module Type Upgrade-Result ------ ------------------ -------------- 1 SUP Success EPLDs upgraded. Module 1 EPLD upgrade is successful. cs2# -
After the switch reboot, log in to verify that the new version of EPLD loaded successfully.
Show example
cs2# show version module 1 epld EPLD Device Version --------------------------------------- MI FPGA 0x12 IO FPGA 0x12
-
Verify the health of cluster ports on the cluster.
-
Verify that cluster ports are up and healthy across all nodes in the cluster:
network port show -role clusterShow example
cluster1::*> network port show -role cluster Node: cluster1-01 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ----------- -------- ------ e0a Cluster Cluster up 9000 auto/10000 healthy false e0b Cluster Cluster up 9000 auto/10000 healthy false Node: cluster1-02 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ----------- -------- ------ e0a Cluster Cluster up 9000 auto/10000 healthy false e0b Cluster Cluster up 9000 auto/10000 healthy false Node: cluster1-03 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ----------- -------- ------ e0a Cluster Cluster up 9000 auto/100000 healthy false e0d Cluster Cluster up 9000 auto/100000 healthy false Node: cluster1-04 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ----------- -------- ------ e0a Cluster Cluster up 9000 auto/100000 healthy false e0d Cluster Cluster up 9000 auto/100000 healthy false 8 entries were displayed. -
Verify the switch health from the cluster.
network device-discovery show -protocol cdpShow example
cluster1::*> network device-discovery show -protocol cdp Node/ Local Discovered Protocol Port Device (LLDP: ChassisID) Interface Platform ----------- ------ ------------------------- ----------------- ---------- cluster1-01/cdp e0a cs1 Ethernet1/7 N3K-C3232C e0d cs2 Ethernet1/7 N3K-C3232C cluster01-2/cdp e0a cs1 Ethernet1/8 N3K-C3232C e0d cs2 Ethernet1/8 N3K-C3232C cluster01-3/cdp e0a cs1 Ethernet1/1/1 N3K-C3232C e0b cs2 Ethernet1/1/1 N3K-C3232C cluster1-04/cdp e0a cs1 Ethernet1/1/2 N3K-C3232C e0b cs2 Ethernet1/1/2 N3K-C3232C cluster1::*> system cluster-switch show -is-monitoring-enabled-operational true Switch Type Address Model --------------------------- ------------------ ---------------- ---------- cs1 cluster-network 10.233.205.90 N3K-C3232C Serial Number: FOCXXXXXXGD Is Monitored: true Reason: None Software Version: Cisco Nexus Operating System (NX-OS) Software, Version 9.3(5) Version Source: CDP cs2 cluster-network 10.233.205.91 N3K-C3232C Serial Number: FOCXXXXXXGS Is Monitored: true Reason: None Software Version: Cisco Nexus Operating System (NX-OS) Software, Version 9.3(5) Version Source: CDP 2 entries were displayed.You might observe the following output on the cs1 switch console depending on the RCF version previously loaded on the switch:
2020 Nov 17 16:07:18 cs1 %$ VDC-1 %$ %STP-2-UNBLOCK_CONSIST_PORT: Unblocking port port-channel1 on VLAN0092. Port consistency restored. 2020 Nov 17 16:07:23 cs1 %$ VDC-1 %$ %STP-2-BLOCK_PVID_PEER: Blocking port-channel1 on VLAN0001. Inconsistent peer vlan. 2020 Nov 17 16:07:23 cs1 %$ VDC-1 %$ %STP-2-BLOCK_PVID_LOCAL: Blocking port-channel1 on VLAN0092. Inconsistent local vlan.
-
-
Verify that the cluster is healthy:
cluster showShow example
cluster1::*> cluster show Node Health Eligibility Epsilon -------------------- -------- ------------- ------- cluster1-01 true true false cluster1-02 true true false cluster1-03 true true true cluster1-04 true true false 4 entries were displayed. cluster1::*>
-
Repeat steps 6 to 14 on switch cs1.
-
Enable auto-revert on the cluster LIFs.
network interface modify -vserver Cluster -lif * -auto-revert true -
Verify that the cluster LIFs have reverted to their home port:
network interface show -role clusterShow example
cluster1::*> network interface show -role cluster Logical Status Network Current Current Is Vserver Interface Admin/Oper Address/Mask Node Port Home ----------- ------------------ ---------- ------------------ ------------------- ------- ---- Cluster cluster1-01_clus1 up/up 169.254.3.4/23 cluster1-01 e0d true cluster1-01_clus2 up/up 169.254.3.5/23 cluster1-01 e0d true cluster1-02_clus1 up/up 169.254.3.8/23 cluster1-02 e0d true cluster1-02_clus2 up/up 169.254.3.9/23 cluster1-02 e0d true cluster1-03_clus1 up/up 169.254.1.3/23 cluster1-03 e0b true cluster1-03_clus2 up/up 169.254.1.1/23 cluster1-03 e0b true cluster1-04_clus1 up/up 169.254.1.6/23 cluster1-04 e0b true cluster1-04_clus2 up/up 169.254.1.7/23 cluster1-04 e0b true 8 entries were displayed. cluster1::*>If any cluster LIFs have not returned to their home ports, revert them manually from the local node:
network interface revert -vserver Cluster -lif <lif_name>
What's next?