Upgrade the RCF version - Cisco Nexus 9364D-GX2A
Suggest changes
PDFs
You upgrade the Reference Configuration File (RCF) version on your Cisco Nexus 9364D-GX2A switches to support new hardware, a new ONTAP version, or to align with a new NX-OS version in your AFX 1K or AFX 2K system configuration.
|
Do not upgrade or downgrade your RCF version to an unsupported version. |
-
You must have read the Install or upgrade RCF overview page.
-
Make sure you have a current backup of the switch configuration.
-
Verify that your cluster is fully functional, meaning that there are no errors in the logs or similar issues.
-
You must have the current RCF version.
-
You need a boot configuration in the RCF that reflects the desired boot images.
If you need to change the boot configuration to reflect the current boot images, you must do so before reapplying the RCF so that the correct version is instantiated on future reboots.
-
To ensure nondisruptive cluster operations, this procedure has you migrate all of the cluster LIFs to the operational partner switch while performing the steps on the target switch.
-
No operational inter-switch link (ISL) is needed during this procedure. This is by design because RCF version changes can affect ISL connectivity temporarily.
-
Because this procedure requires the use of ONTAP commands and Cisco Nexus 9000 Series Switches commands, the steps indicate which CLI you are using to enter commands.
-
The examples in this procedure use the following switch and node nomenclature:
-
The names of the two Cisco switches are cs1 and cs2.
-
The node names are node1-01, node1-02, node1-03, and node1-04.
-
The cluster LIF names are:
-
node1-01_clus1 and node1-01_clus2 for node1-01
-
node1-02_clus1 and node1-02_clus2 for node1-02
-
node1-03_clus1 and node1-03_clus2 for node1-03
-
node1-04_clus1 and node1-04_clus2 for node1-04
-
-
The
cluster1::*>prompt indicates the name of the cluster.
-
-
The examples shown in this procedure are for AFX 1K systems with switches running RCF 10.0. If you have an AFX 2K system, the input and output can vary.
Step 1: Prepare for the upgrade
Using the ONTAP CLI for the following steps, prepare for the RCF upgrade by suppressing automatic case creation as needed, checking administrative and operational status on the ports, and disabling auto-revert on the cluster LIFs.
-
If AutoSupport is enabled on this cluster, suppress automatic case creation by invoking an AutoSupport message:
system node autosupport invoke -node * -type all -message MAINT=<number of hours down>h -
Change the privilege level to advanced, entering y when prompted to continue:
set -privilege advancedThe advanced prompt (*>) appears.
-
Display the cluster ports on each node that are connected to the switches:
network device-discovery showShow example
cluster1::*> network device-discovery show Node/ Local Discovered Protocol Port Device (LLDP: ChassisID) Interface Platform ----------- ------ ------------------------- ---------------- -------- node1-01/cdp e10a cs1(FLMXXXXXXXX) Ethernet1/16/3 N9K-C9364D-GX2A e10b cs1(FDOXXXXXXXX) Ethernet1/13/3 N9K-C9364D-GX2A e11a cs1(FLMXXXXXXXX) Ethernet1/16/4 N9K-C9364D-GX2A e11b cs1(FDOXXXXXXXX) Ethernet1/13/4 N9K-C9364D-GX2A e1a cs1(FLMXXXXXXXX) Ethernet1/16/1 N9K-C9364D-GX2A e1b cs1(FDOXXXXXXXX) Ethernet1/13/1 N9K-C9364D-GX2A . . . e7a cs1(FLMXXXXXXXX) Ethernet1/16/2 N9K-C9364D-GX2A e7b cs1(FDOXXXXXXXX) Ethernet1/13/2 N9K-C9364D-GX2A node1-01/lldp e10a cs1 (c8:60:8f:xx:xx:xx) Ethernet1/16/3 - e10b cs2 (04:e3:87:xx:xx:xx) Ethernet1/16/3 - e11a cs1 (c8:60:8f:xx:xx:xx) Ethernet1/16/4 - e11b cs2 (04:e3:87:xx:xx:xx) Ethernet1/16/4 - e1a cs1 (c8:60:8f:xx:xx:xx) Ethernet1/16/1 - e1b cs2 (04:e3:87:xx:xx:xx) Ethernet1/16/1 - . . . e7a cs1 (c8:60:8f:xx:xx:xx) Ethernet1/16/2 - e7b cs2 (04:e3:87:xx:xx:xx) Ethernet1/16/2 - . . . -
Check the administrative and operational status of each port.
-
Cluster ports
-
Verify that all the cluster ports are up with a healthy status:
network port show -ipspace ClusterShow example
cluster1::*> network port show -ipspace Cluster Node: node1-01 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ----------- -------- ------ e7a Cluster Cluster up 9000 auto/100000 healthy false e7b Cluster Cluster up 9000 auto/100000 healthy false Node:node1-02 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ----------- -------- ------ e7a Cluster Cluster up 9000 auto/100000 healthy false e7b Cluster Cluster up 9000 auto/100000 healthy false Node: node1-03 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ----------- -------- ------ e7a Cluster Cluster up 9000 auto/10000 healthy false e7b Cluster Cluster up 9000 auto/10000 healthy false Node: node1-04 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ----------- -------- ------ e7a Cluster Cluster up 9000 auto/10000 healthy false e7b Cluster Cluster up 9000 auto/10000 healthy false cluster1::*> -
Verify that all the cluster interfaces (LIFs) are on the home port:
network interface show -role clusterShow example
cluster1::*> network interface show -role cluster Logical Status Network Current Current Is Vserver Interface Admin/Oper Address/Mask Node Port Home ----------- ---------------- ----------- ----------------- ------------ ------- ---- Cluster node1-01_clus1 up/up 169.254.3.4/23 node1-01 e7a true node1-01_clus2 up/up 169.254.3.5/23 node1-01 e7b true node1-02_clus1 up/up 169.254.3.8/23 node1-02 e7a true node1-02_clus2 up/up 169.254.3.9/23 node1-02 e7b true node1-03_clus1 up/up 169.254.1.3/23 node1-03 e7a true node1-03_clus2 up/up 169.254.1.1/23 node1-03 e7b true node1-04_clus1 up/up 169.254.1.6/23 node1-04 e7a true node1-04_clus2 up/up 169.254.1.7/23 node1-04 e7b true 8 entries were displayed. -
Verify that the cluster displays information for both switches:
system cluster-switch show -is-monitoring-enabled-operational trueShow example
cluster1::*> system cluster-switch show -is-monitoring-enabled-operational true Switch Type Address Model --------------------------- ------------------ ---------------- ----- cs1(FDOXXXXXXXX) cluster-network 10.228.137.233 N9K-C9364D-GX2A Serial Number: FDOXXXXXXXX Is Monitored: true Reason: None Software Version: Cisco Nexus Operating System (NX-OS) Software, Version 10.4(2) Version Source: CDP/ISDP cs2(FLMXXXXXXXX) cluster-network 10.228.137.234 N9K-C9364D-GX2A Serial Number: FLMXXXXXXXX Is Monitored: true Reason: None Software Version: Cisco Nexus Operating System (NX-OS) Software, Version 10.4(2) Version Source: CDP/ISDP
-
-
HA ports
-
Verify that all the HA ports are up with a healthy status:
ha interconnect status show -node <node-name>Show example
cluster1::*> ha interconnect status show -node node1-01 (system ha interconnect status show) Node: node1-01 Link 0 Status: up Link 1 Status: up Is Link 0 Active: true Is Link 1 Active: true IC RDMA Connection: up Slot: 0 Debug Firmware: no Interconnect Port 0 : Port Name: e1a-17 MTU: 4096 Link Information: ACTIVE Interconnect Port 1 : Port Name: e1b-18 MTU: 4096 Link Information: ACTIVE cluster1::*> ha interconnect status show -node node1-01 (system ha interconnect status show) Node: node1-01 Link 0 Status: up Link 1 Status: up Is Link 0 Active: true Is Link 1 Active: true IC RDMA Connection: up Slot: 0 Debug Firmware: no Interconnect Port 0 : Port Name: e1a-17 MTU: 4096 Link Information: ACTIVE Interconnect Port 1 : Port Name: e1b-18 MTU: 4096 Link Information: ACTIVE . . .
-
-
Storage ports
-
Verify that all the storage ports are up with a healthy status:
storage port show -port-type ENETShow example
cluster1::*> storage port show -port-type ENET Speed Node Port Type Mode (Gb/s) State Status ------------------ ---- ----- ------- ------ -------- ----------- node1-01 e10a ENET - 100 enabled online e10b ENET - 100 enabled online e11a ENET - 100 enabled online e11b ENET - 100 enabled online node1-02 e10a ENET - 100 enabled online e10b ENET - 100 enabled online e11a ENET - 100 enabled online e11b ENET - 100 enabled online node1-03 e10a ENET - 100 enabled online e10b ENET - 100 enabled online e11a ENET - 100 enabled online node1-04 e10a ENET - 100 enabled online e10b ENET - 100 enabled online e11a ENET - 100 enabled online e11b ENET - 100 enabled online 16 entries were displayed.
-
-
Storage shelf ports
-
Verify that all the storage shelf ports are up with a healthy status:
storage shelf port showShow example
cluster1::*> storage shelf port show Shelf ID Module State Internal? ----- -- ------ ------------ --------- 1.1 0 A connected false 1 A connected false 2 A connected false 3 A connected false 4 A connected false 5 A connected false 6 A connected false 7 A connected false 8 B connected false 9 B connected false 10 B connected false 11 B connected false 12 B connected false 13 B connected false 14 B connected false 15 B connected false 16 entries were displayed. -
Verify that all the storage shelf ports are up with a healthy status:
storage shelf port show -fields remote-device,remote-port,connector-stateShow example
cluster1::*> storage shelf port show -fields remote-device,remote-port,connector-state shelf id connector-state remote-port remote-device ----- -- --------------- -------------- ----------------- 1.1 0 connected Ethernet1/17/1 CX9332D-cs1 1.1 1 connected Ethernet1/15/1 CX9364D-cs1 1.1 2 connected Ethernet1/17/2 CX9332D-cs1 1.1 3 connected Ethernet1/15/2 CX9364D-cs1 1.1 4 connected Ethernet1/17/3 CX9332D-cs1 1.1 5 connected Ethernet1/15/3 CX9364D-cs1 1.1 6 connected Ethernet1/17/4 CX9332D-cs1 1.1 7 connected Ethernet1/15/4 CX9364D-cs1 1.1 8 connected Ethernet1/19/1 CX9332D-cs1 1.1 9 connected Ethernet1/17/1 CX9364D-cs1 1.1 10 connected Ethernet1/19/2 CX9332D-cs1 1.1 11 connected Ethernet1/17/2 CX9364D-cs1 1.1 12 connected Ethernet1/19/3 CX9332D-cs1 1.1 13 connected Ethernet1/17/3 CX9364D-cs1 1.1 14 connected Ethernet1/19/4 CX9332D-cs1 1.1 15 connected Ethernet1/17/4 CX9364D-cs1 16 entries were displayed.
-
-
-
Disable auto-revert on the cluster LIFs. The cluster LIFs fail over to the partner cluster switch and remain there as you perform this procedure on the targeted switch:
network interface modify -vserver Cluster -lif * -auto-revert false
Step 2: Upgrade the RCF
You use the switch CLI and the ONTAP CLI (as indicated in the following steps) for the RCF upgrade. You upgrade the RCF on the first switch (cs1) and then repeat these steps for the second switch (cs2).
-
Using the switch CLI, on switch cs1, shut down the ports connected to all the ports of the nodes.
Make sure to shutdown all connected ports to avoid any network connection issues. Refer to the Knowledge Base article Node out of quorum when migrating cluster LIF during switch OS upgrade for further details. cs1# config cs1(config)# interface e1/1/1-4,e1/2/1-4,e1/3/1-4,e1/4/1-4,e1/5/1-4,e1/6/1-4,e1/7/1-4,e1/8/1-4 cs1(config-if-range)# shutdown cs1(config)# interface e1/9/1-4,e1/10/1-4,e1/11/1-4,e1/12/1-4,e1/13/1-4,e1/14/1-4,e1/15/1-4,e1/16/1-4 cs1(config-if-range)# shutdown cs1(config)# interface e1/17/1-4,e1/18/1-4,e1/19/1-4,e1/20/1-4,e1/21/1-4,e1/22/1-4,e1/23/1-4,e1/24/1-4 cs1(config-if-range)# shutdown cs1(config)# interface e1/25/1-4,e1/26/1-4,e1/27/1-4,e1/28/1-4,e1/29/1-4,e1/30/1-4,e1/31/1-4,e1/32/1-4 cs1(config-if-range)# shutdown cs1(config)# interface e1/33/1-4,e1/34/1-4,e1/35/1-4,e1/36/1-4,e1/37/1-4,e1/38/1-4,e1/39/1-4,e1/40/1-4 cs1(config-if-range)# shutdown cs1(config)# interface e1/41/1-4,e1/42/1-4,e1/43/1-4,e1/44/1-4,e1/35/1-4,e1/46/1-4,e1/47/1-4,e1/48/1-4 cs1(config-if-range)# shutdown cs1(config)# interface e1/49/1-4,e1/50/1-4,e1/51/1-4,e1/52/1-4,e1/53/1-4,e1/54/1-4,e1/55/1-4,e1/56/1-4 cs1(config-if-range)# shutdown cs1(config)# interface e1/57/1-4,e1/58/1-4,e1/59/1-4,e1/60/1-4,e1/61/1-4,e1/62/1-4 cs1(config-if-range)# shutdown cs1(config-if-range)# exit cs1(config)# exit
-
Using the ONTAP CLI, verify that the cluster LIFs have failed over to the ports hosted on switch cs1. This might take a few seconds.
network interface show -role clusterShow example
cluster1::*> network interface show -role cluster Logical Status Network Current Current Is Vserver Interface Admin/Oper Address/Mask Node Port Home ----------- ----------------- ---------- ------------------ ------------- ------- ---- Cluster node1-01_clus1 up/up 169.254.36.44/16 node1-01 e7a true node1-01_clus2 up/up 169.254.7.5/16 node1-01 e7b true node1-02_clus1 up/up 169.254.197.206/16 node1-02 e7a true node1-02_clus2 up/up 169.254.195.186/16 node1-02 e7b true node1-03_clus1 up/up 169.254.192.49/16 node1-03 e7a true node1-03_clus2 up/up 169.254.182.76/16 node1-03 e7b true node1-04_clus1 up/up 169.254.59.49/16 node1-04 e7a true node1-04_clus2 up/up 169.254.62.244/16 node1-04 e7b true 8 entries were displayed. -
Using the ONTAP CLI, verify that the cluster is healthy:
cluster showShow example
cluster1::*> cluster show Node Health Eligibility Epsilon ----------------- ------- ------------ ------- node1-01 true true false node1-02 true true false node1-03 true true true node1-04 true true false 4 entries were displayed.
-
If you have not already done so, using the switch CLI, save a copy of the current switch configuration by copying the output of the following command to a text file:
show running-config-
Use the
show diff running-configcommand to compare with the saved RCF file in the bootflash. Otherwise, use a third-party diff/compare tool.
-
-
Save basic configuration details to the
write_erase.cfgfile on the bootflash.
Make sure to configure the following:
-
Username and password
-
Management IP address
-
Default gateway
-
Switch name
If the
write_erase.cfgfile already exists, rename it towrite_erase_old.cfg, as follows:cs1# move bootflash:write_erase.cfg write_erase_old.cfgcs1# show run | i "username admin password" > bootflash:write_erase.cfgcs1# show run | section "vrf context management" >> bootflash:write_erase.cfgcs1# show run | section "interface mgmt0" >> bootflash:write_erase.cfgcs1# show run | section "switchname" >> bootflash:write_erase.cfgcs1# echo "hardware access-list tcam region ing-racl 1024" >> bootflash:write_erase.cfgcs1# echo "hardware access-list tcam region egr-racl 1024" >> bootflash:write_erase.cfgcs1# echo "hardware access-list tcam region ing-l2-qos 1536" >> bootflash:write_erase.cfgRefer to the Knowledge Base article How to clear configuration on a Cisco interconnect switch while retaining remote connectivity for further details.
-
-
Using the switch CLI, verify that the
write_erase.cfgfile is populated as expected:show file bootflash:write_erase.cfg -
Using the switch CLI, erase the current saved configuration:
write eraseWarning: This command will erase the startup-configuration.Do you wish to proceed anyway? (y/n) [n] y -
Using the switch CLI, copy the previously saved basic configuration into the startup configuration:
copy bootflash:write_erase.cfg startup-config -
Using the switch CLI, reboot the switch:
reloadThis command will reboot the system. (y/n)? [n] y -
After the management IP address is reachable again, log in to the switch through SSH.
You might need to update host file entries related to the SSH keys.
-
Using the switch CLI, copy the RCF to the bootflash of switch cs1 using one of the following transfer protocols: FTP, TFTP, SFTP, or SCP.
For more information on Cisco commands, refer to the appropriate guide in the Cisco Nexus 9000 Series NX-OS Command Reference.
copy <protocol>: bootflash: vrf managementShow example
This example shows TFTP being used to copy an RCF to the bootflash on switch cs1:
cs1# copy tftp: bootflash: vrf management Enter source filename: NX9364D-GX2A-RCF-v10.0-Shared.txt Enter hostname for the tftp server: 172.22.201.50 Trying to connect to tftp server......Connection to Server Established. TFTP get operation was successful Copy complete, now saving to disk (please wait)...
-
Using the switch CLI, apply the RCF previously downloaded to the bootflash.
For more information on Cisco commands, refer to the appropriate guide in the Cisco Nexus 9000 Series NX-OS Command Reference.
This example shows the RCF file
NX9364D-GX2A-RCF-v10.0-Shared.txtbeing installed on switch cs1:cs1# copy NX9364D-GX2A-RCF-v10.0-Shared.txt running-config echo-commands
Make sure to thoroughly read the Installation notes, Important Notes, and banner sections of your RCF.
You must read and follow these instructions to ensure the proper configuration and operation of the switch.
-
Using the switch CLI, verify that the RCF file is the correct newer version:
show running-configWhen you check the output to verify you have the correct RCF, make sure that the following information is correct:
-
The RCF banner
-
The node and port settings
-
Customizations
The output varies according to your site configuration. Check the port settings and refer to the release notes for any changes specific to the RCF that you have installed.
-
-
Reapply any previous customizations to the switch configuration.
If your switches are running RCF 10.1 or later, you can change the port speed configuration if needed.
-
Using the switch CLI, after you verify the RCF versions, custom additions, and switch settings are correct, copy the running-config file to the startup-config file.
For more information on Cisco commands, refer to the appropriate guide in the Cisco Nexus 9000 Series NX-OS Command Reference.
copy running-config startup-config[] 100% Copy complete -
Using the switch CLI, reboot switch cs1. You can ignore the “cluster switch health monitor” alerts and “cluster ports down” events reported on the nodes while the switch reboots.
reloadThis command will reboot the system. (y/n)? [n] y -
Using the ONTAP CLI, verify the health of all the ports on the cluster:
-
Cluster ports
-
Verify that cluster ports are up and healthy across all nodes in the cluster:
network port show -ipspace ClusterShow example
cluster1::*> network port show -ipspace Cluster Node: node1-01 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ----------- -------- ------ e7a Cluster Cluster up 9000 auto/10000 healthy false e7b Cluster Cluster up 9000 auto/10000 healthy false Node: node1-02 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ----------- -------- ------ e7a Cluster Cluster up 9000 auto/10000 healthy false e7b Cluster Cluster up 9000 auto/10000 healthy false Node: node1-03 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ----------- -------- ------ e7a Cluster Cluster up 9000 auto/100000 healthy false e7b Cluster Cluster up 9000 auto/100000 healthy false Node: node1-04 Ignore Speed(Mbps) Health Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status --------- ------------ ---------------- ---- ---- ----------- -------- ------ e7a Cluster Cluster up 9000 auto/100000 healthy false e7b Cluster Cluster up 9000 auto/100000 healthy false 8 entries were displayed. -
Verify the switch health from the cluster.
network device-discovery show -protocol cdpsystem cluster-switch show -is-monitoring-enabled-operational trueShow example
cluster1::*> network device-discovery show -protocol cdp Node/ Local Discovered Protocol Port Device (LLDP: ChassisID) Interface Platform ----------- ------ ------------------------- ----------------- -------- node1-01/cdp e10a cs1(FLMXXXXXXXX) Ethernet1/16/3 N9K-C9364D-GX2A e10b cs2(FDOXXXXXXXX) Ethernet1/13/3 N9K-C9364D-GX2A e11a cs1(FLMXXXXXXXX) Ethernet1/16/4 N9K-C9364D-GX2A e11b cs2(FDOXXXXXXXX) Ethernet1/13/4 N9K-C9364D-GX2A e1a cs1(FLMXXXXXXXX) Ethernet1/16/1 N9K-C9364D-GX2A e1b cs2(FLMXXXXXXXX) Ethernet1/13/1 N9K-C9364D-GX2A . . . e7a cs1(FLMXXXXXXXX) Ethernet1/16/2 N9K-C9364D-GX2A e7b cs2(FDOXXXXXXXX) Ethernet1/13/2 N9K-C9364D-GX2A node1-02/cdp . . . cluster1::*> system cluster-switch show -is-monitoring-enabled-operational true Switch Type Address Model --------------------------- ------------------ ---------------- ----- cs2(FDOXXXXXXXX) cluster-network 10.228.137.233 N9K-C9364D-GX2A Serial Number: FDOXXXXXXXX Is Monitored: true Reason: None Software Version: Cisco Nexus Operating System (NX-OS) Software, Version 10.4(2) Version Source: CDP/ISDP cs1(FLMXXXXXXXX) cluster-network 10.228.137.234 N9K-C9364D-GX2A Serial Number: FLMXXXXXXXX Is Monitored: true Reason: None Software Version: Cisco Nexus Operating System (NX-OS) Software, Version 10.4(2) Version Source: CDP/ISDP 2 entries were displayed.
-
-
HA ports
-
Verify that all the HA ports are up with a healthy status:
ha interconnect status show -node <node-name>Show example
cluster1::*> ha interconnect status show -node node1-01 (system ha interconnect status show) Node: node1-01 Link 0 Status: up Link 1 Status: up Is Link 0 Active: true Is Link 1 Active: true IC RDMA Connection: up Slot: 0 Debug Firmware: no Interconnect Port 0 : Port Name: e1a-17 MTU: 4096 Link Information: ACTIVE Interconnect Port 1 : Port Name: e1b-18 MTU: 4096 Link Information: ACTIVE cluster1::*> ha interconnect status show -node node1-02 (system ha interconnect status show) Node: node1-02 Link 0 Status: up Link 1 Status: up Is Link 0 Active: true Is Link 1 Active: true IC RDMA Connection: up Slot: 0 Debug Firmware: no Interconnect Port 0 : Port Name: e1a-17 MTU: 4096 Link Information: ACTIVE Interconnect Port 1 : Port Name: e1b-18 MTU: 4096 Link Information: ACTIVE . . .
-
-
Storage ports
-
Verify that all the storage ports are up with a healthy status:
storage port show -port-type ENETShow example
cluster1::*> storage port show -port-type ENET Speed Node Port Type Mode (Gb/s) State Status ------------------ ---- ----- ------- ------ -------- ----------- node1-01 e10a ENET - 100 enabled online e10b ENET - 100 enabled online e11a ENET - 100 enabled online e11b ENET - 100 enabled online node1-02 e10a ENET - 100 enabled online e10b ENET - 100 enabled online e11a ENET - 100 enabled online e11b ENET - 100 enabled online node1-03 e10a ENET - 100 enabled online e10b ENET - 100 enabled online e11a ENET - 100 enabled online node1-04 e10a ENET - 100 enabled online e10b ENET - 100 enabled online e11a ENET - 100 enabled online e11b ENET - 100 enabled online 16 entries were displayed.
-
-
Storage shelf ports
-
Verify that all the storage shelf ports are up with a healthy status:
storage shelf port showShow example
cluster1::*> storage shelf port show Shelf ID Module State Internal? ----- -- ------ ------------ --------- 1.1 0 A connected false 1 A connected false 2 A connected false 3 A connected false 4 A connected false 5 A connected false 6 A connected false 7 A connected false 8 B connected false 9 B connected false 10 B connected false 11 B connected false 12 B connected false 13 B connected false 14 B connected false 15 B connected false 16 entries were displayed. -
Verify the connection status of all the storage shelf ports:
storage shelf port show -fields remote-device,remote-port,connector-stateShow example
cluster1::*> storage shelf port show -fields remote-device,remote-port,connector-state shelf id connector-state remote-port remote-device ----- -- --------------- -------------- ----------------- 1.1 0 connected Ethernet1/17/1 CX9332D-cs1 1.1 1 connected Ethernet1/15/1 CX9364D-cs1 1.1 2 connected Ethernet1/17/2 CX9332D-cs1 1.1 3 connected Ethernet1/15/2 CX9364D-cs1 1.1 4 connected Ethernet1/17/3 CX9332D-cs1 1.1 5 connected Ethernet1/15/3 CX9364D-cs1 1.1 6 connected Ethernet1/17/4 CX9332D-cs1 1.1 7 connected Ethernet1/15/4 CX9364D-cs1 1.1 8 connected Ethernet1/19/1 CX9332D-cs1 1.1 9 connected Ethernet1/17/1 CX9364D-cs1 1.1 10 connected Ethernet1/19/2 CX9332D-cs1 1.1 11 connected Ethernet1/17/2 CX9364D-cs1 1.1 12 connected Ethernet1/19/3 CX9332D-cs1 1.1 13 connected Ethernet1/17/3 CX9364D-cs1 1.1 14 connected Ethernet1/19/4 CX9332D-cs1 1.1 15 connected Ethernet1/17/4 CX9364D-cs1 16 entries were displayed.
-
-
-
Using the ONTAP CLI, verify that the cluster is healthy:
cluster showShow example
cluster1::*> cluster show Node Health Eligibility Epsilon ----------------- -------- ------------- ------- node1-01 true true false node1-02 true true false node1-03 true true true node1-04 true true false 4 entries were displayed.
-
Repeat steps 4 to 18 on switch cs2.
-
Using the ONTAP CLI, enable auto-revert on the cluster LIFs.
network interface modify -vserver Cluster -lif * -auto-revert true
Step 3: Verify the cluster network configuration and cluster health
After the RCF is upgraded on both switches, verify the cluster network configuration and cluster health.
-
Using the switch CLI, verify that the switch ports connected to the cluster ports are up.
show interface briefShow example
cs1# show interface brief | grep up . . Eth1/9/3 1 eth trunk up none 100G(D) -- Eth1/9/4 1 eth trunk up none 100G(D) -- Eth1/15/1 1 eth trunk up none 100G(D) -- Eth1/15/2 1 eth trunk up none 100G(D) -- Eth1/15/3 1 eth trunk up none 100G(D) -- Eth1/15/4 1 eth trunk up none 100G(D) -- Eth1/16/1 1 eth trunk up none 100G(D) -- Eth1/16/2 1 eth trunk up none 100G(D) -- Eth1/16/3 1 eth trunk up none 100G(D) -- Eth1/16/4 1 eth trunk up none 100G(D) -- Eth1/17/1 1 eth trunk up none 100G(D) -- Eth1/17/2 1 eth trunk up none 100G(D) -- Eth1/17/3 1 eth trunk up none 100G(D) -- Eth1/17/4 1 eth trunk up none 100G(D) -- . . .
-
Using the switch CLI, verify that the expected nodes are still connected:
show cdp neighborsShow example
cs1# show cdp neighbors Capability Codes: R - Router, T - Trans-Bridge, B - Source-Route-Bridge S - Switch, H - Host, I - IGMP, r - Repeater, V - VoIP-Phone, D - Remotely-Managed-Device, s - Supports-STP-Dispute Device-ID Local Intrfce Hldtme Capability Platform Port ID cs2(FDOXXXXXXXX) Eth1/63 179 R S I s N9K-C9332D-GX2B Eth1/31 cs2(FDOXXXXXXXX) Eth1/64 179 R S I s N9K-C9332D-GX2B Eth1/32 node1-01 Eth1/4/1 123 H AFX-1K e1a node1-01 Eth1/4/2 123 H AFX-1K e7a node1-01 Eth1/4/3 123 H AFX-1K e10a node1-01 Eth1/4/4 123 H AFX-1K e11a node1-02 Eth1/9/1 138 H AFX-1K e1a node1-02 Eth1/9/2 138 H AFX-1K e7a node1-02 Eth1/9/3 138 H AFX-1K e10a node1-02 Eth1/9/4 138 H AFX-1K e11a node1-03 Eth1/15/1 138 H AFX-1K e1a node1-03 Eth1/15/2 138 H AFX-1K e7a node1-03 Eth1/15/3 138 H AFX-1K e10a node1-03 Eth1/15/4 138 H AFX-1K e11a node1-04 Eth1/16/1 173 H AFX-1K e1a node1-04 Eth1/16/2 173 H AFX-1K e7a node1-04 Eth1/16/3 173 H AFX-1K e10a node1-04 Eth1/16/4 173 H AFX-1K e11a Total entries displayed: 18 -
Using the switch CLI, verify that the cluster nodes are in their correct cluster VLANs using the following commands:
show vlan briefshow interface trunkShow example
cs1# show vlan brief VLAN Name Status Ports ---- -------------------------------- --------- ------------------------------- 1 default active Po1, Po999, Eth1/63, Eth1/64 Eth1/65, Eth1/66, Eth1/1/1 Eth1/1/2, Eth1/1/3, Eth1/1/4 . . . Eth1/61/2, Eth1/61/3, Eth1/61/4 Eth1/62/1, Eth1/62/2, Eth1/62/3 Eth1/62/4 17 VLAN0017 active Eth1/1/1, Eth1/1/2, Eth1/1/3 Eth1/1/4, Eth1/2/1, Eth1/2/2 . . . Eth1/61/1, Eth1/61/2, Eth1/61/3 Eth1/61/4, Eth1/62/1, Eth1/62/2 Eth1/62/3, Eth1/62/4 18 VLAN0018 active Eth1/1/1, Eth1/1/2, Eth1/1/3 Eth1/1/4, Eth1/2/1, Eth1/2/2 . . . Eth1/61/1, Eth1/61/2, Eth1/61/3 Eth1/61/4, Eth1/62/1, Eth1/62/2 Eth1/62/3, Eth1/62/4 30 VLAN0030 active Eth1/1/1, Eth1/1/2, Eth1/1/3 Eth1/1/4, Eth1/2/1, Eth1/2/2 . . . Eth1/61/1, Eth1/61/2, Eth1/61/3 Eth1/61/4, Eth1/62/1, Eth1/62/2 Eth1/62/3, Eth1/62/4 40 VLAN0040 active Eth1/1/1, Eth1/1/2, Eth1/1/3 Eth1/1/4, Eth1/2/1, Eth1/2/2 . . . Eth1/61/1, Eth1/61/2, Eth1/61/3 Eth1/61/4, Eth1/62/1, Eth1/62/2 Eth1/62/3, Eth1/62/4 cs1# show interface trunk ----------------------------------------------------- Port Native Status Port Vlan Channel ----------------------------------------------------- Eth1/1/1 1 trunking -- Eth1/1/2 1 trunking -- Eth1/1/3 1 trunking -- Eth1/1/4 1 trunking -- Eth1/2/1 1 trunking -- Eth1/2/2 1 trunking -- Eth1/2/3 1 trunking -- Eth1/2/4 1 trunking -- . . . Eth1/62/2 none Eth1/62/3 none Eth1/62/4 none Eth1/63 none Eth1/64 none Po1 1
For specific port and VLAN usage details, refer to the banner and important notes section in your RCF. -
Using the switch CLI, verify that the ISL between cs1 and cs2 is functional:
show port-channel summaryShow example
cs1# show port-channel summary Flags: D - Down P - Up in port-channel (members) I - Individual H - Hot-standby (LACP only) s - Suspended r - Module-removed b - BFD Session Wait S - Switched R - Routed U - Up (port-channel) p - Up in delay-lacp mode (member) M - Not in use. Min-links not met -------------------------------------------------------------------------------- Group Port- Type Protocol Member Ports Channel -------------------------------------------------------------------------------- 1 Po1(SU) Eth LACP Eth1/63(P) Eth1/64(P) 999 Po999(SD) Eth NONE -- cs1# -
Using the ONTAP CLI, verify that the cluster LIFs have reverted to their home port:
network interface show -role clusterShow example
cluster1::*> network interface show -role cluster Logical Status Network Current Current Is Vserver Interface Admin/Oper Address/Mask Node Port Home ----------- ------------------ ---------- ------------------ ------------ ------- ---- Cluster node1-01_clus1 up/up 169.254.36.44/16 node1-01 e7a true node1-01_clus2 up/up 169.254.7.5/16 node1-01 e7b true node1-02_clus1 up/up 169.254.197.206/16 node1-02 e7a true node1-02_clus2 up/up 169.254.195.186/16 node1-02 e7b true node1-03_clus1 up/up 169.254.192.49/16 node1-03 e7a true node1-03_clus2 up/up 169.254.182.76/16 node1-03 e7b true node1-04_clus1 up/up 169.254.59.49/16 node1-04 e7a true node1-04_clus2 up/up 169.254.62.244/16 node1-04 e7b true 8 entries were displayed.If any cluster LIFs have not returned to their home ports, revert them manually from the local node:
network interface revert -vserver vserver_name -lif <lif-name> -
Using the ONTAP CLI, verify that the cluster is healthy:
cluster showShow example
cluster1::*> cluster show Node Health Eligibility Epsilon ----------------- ------- ------------- ------- node1-01 true true false node1-02 true true false node1-03 true true true node1-04 true true false 4 entries were displayed.
-
Using the ONTAP CLI, verify the connectivity of the remote cluster interfaces:
-
You can use the following command to display the details of an accessibility check for cluster connectivity:
network interface check cluster-connectivity showShow example
cluster1::*> network interface check cluster-connectivity show Source Destination Packet Node Date LIF LIF Loss ------ -------------------------- ---------------- --------------- ----------- node1-01 6/4/2025 03:13:33 -04:00 node1-01_clus2 node1-02_clus1 none 6/4/2025 03:13:34 -04:00 node1-01_clus2 node1-02_clus1 none node1-02 6/4/2025 03:13:33 -04:00 node1-02_clus2 node1-01_clus1 none 6/4/2025 03:13:34 -04:00 node1-02_clus2 node1-01_clus2 none . . . -
Alternatively, you can use the following command to check the connectivity:
cluster ping-cluster -node <node-name>Show example
cluster1::*> cluster ping-cluster -node local Host is node1-04 Getting addresses from network interface table... Cluster node1-01_clus1 169.254.36.44 node1-01 e7a Cluster node1-01_clus2 169.254.7.5 node1-01 e7b Cluster node1-02_clus1 169.254.197.206 node1-02 e7a Cluster node1-02_clus2 169.254.195.186 node1-02 e7b Cluster node1-03_clus1 169.254.192.49 node1-03 e7a Cluster node1-03_clus2 169.254.182.76 node1-03 e7b Cluster node1-04_clus1 169.254.59.49 node1-04 e7a Cluster node1-04_clus2 169.254.62.244 node1-04 e7b Local = 169.254.59.49 169.254.62.244 Remote = 169.254.36.44 169.254.7.5 169.254.197.206 169.254.195.186 169.254.192.49 169.254.182.76 Cluster Vserver Id = 4294967293 Ping status: ............ Basic connectivity succeeds on 12 path(s) Basic connectivity fails on 0 path(s) ................................................ Detected 9000 byte MTU on 12 path(s): Local 169.254.59.49 to Remote 169.254.182.76 Local 169.254.59.49 to Remote 169.254.192.49 Local 169.254.59.49 to Remote 169.254.195.186 Local 169.254.59.49 to Remote 169.254.197.206 Local 169.254.59.49 to Remote 169.254.36.44 Local 169.254.59.49 to Remote 169.254.7.5 Local 169.254.62.244 to Remote 169.254.182.76 Local 169.254.62.244 to Remote 169.254.192.49 Local 169.254.62.244 to Remote 169.254.195.186 Local 169.254.62.244 to Remote 169.254.197.206 Local 169.254.62.244 to Remote 169.254.36.44 Local 169.254.62.244 to Remote 169.254.7.5 Larger than PMTU communication succeeds on 12 path(s) RPC status: 6 paths up, 0 paths down (tcp check) 6 paths up, 0 paths down (udp check)
-
After you've upgraded the RCF on your switches, you verify SSH for log collection.