Install the RCF script on NVIDIA SN2100 storage switches
Install the reference configuration file (RCF) script on NVIDIA SN2100 storage switches to configure port assignments and network settings.
Before installing the RCF script, make sure that the following are available on the switch:
-
Supported version of Cumulus Linux is installed.
-
Management port IP address, subnet mask, and default gateway defined via DHCP or manually configured.
Select the appropriate storage RCF script version based on your current ONTAP and Cumulus Linux version. Download RCFs from NVIDIA SN2100 software download page.
The following example procedure shows how to download and apply the RCF script for Storage switches.
Step 1: Install the RCF on the switch
-
Display the available interfaces on the SN2100 switch:
nv show interfaceShow example
cumulus@cumulus:mgmt:~$ nv show interface State Name Spd MTU Mode LLDP Summary ----- ----- --- ----- ----------- ------------------ -------------- ... ... ADMDN swp1 N/A 9216 NotConfigured ADMDN swp2 N/A 9216 NotConfigured ADMDN swp3 N/A 9216 NotConfigured ADMDN swp4 N/A 9216 NotConfigured ADMDN swp5 N/A 9216 NotConfigured ADMDN swp6 N/A 9216 NotConfigured ADMDN swp7 N/A 9216 NotConfigured ADMDN swp8 N/A 9216 NotConfigured ADMDN swp9 N/A 9216 NotConfigured ADMDN swp10 N/A 9216 NotConfigured ADMDN swp11 N/A 9216 NotConfigured ADMDN swp12 N/A 9216 NotConfigured ADMDN swp13 N/A 9216 NotConfigured ADMDN swp14 N/A 9216 NotConfigured ADMDN swp15 N/A 9216 NotConfigured ADMDN swp16 N/A 9216 NotConfigured
-
Copy the RCF python script to the switch:
admin@sw1:mgmt:~$ pwd /home/cumulus cumulus@cumulus:mgmt:~$ cd /tmp cumulus@cumulus:mgmt:/tmp$ scp <user>@<host:/<path>/RCF-v1.14-MSN2100-ST1 ssologin@10.233.204.71's password: RCF-v1.14-MSN2100-ST1 100% 8607 111.2KB/s 00:00
-
Apply the RCF python script RCF-v1.14-MSN2100-ST1:
cumulus@cumulus:mgmt:/tmp$ RCF-v1.14-MSN2100-ST1 [sudo] password for cumulus: ... Step 1: Creating the banner file Step 2: Registering banner message Step 3: Updating the MOTD file Step 4: Ensuring passwordless use of cl-support command by admin Step 5: Disabling apt-get Step 6: Creating the interfaces Step 7: Adding the interface config Step 8: Disabling cdp Step 9: Adding the lldp config Step 10: Adding the RoCE base config Step 11: Modifying RoCE Config Step 12: Configure SNMP Step 13: Reboot the switch
The RCF script completes the steps listed above.
For any RCF python script issues that cannot be corrected, contact NetApp Support for assistance. -
Reapply any previous customizations to the switch configuration. Refer to Review cabling and configuration considerations for details of any further changes required.
-
Connect the controller storage ports and NS224 shelf ports to the switches.
Step 2: Verify the switch and storage connectivity
-
Verify the configuration after the reboot:
nv show interfaceShow example
cumulus@cumulus:mgmt:~$ nv show interface Interface Admin Status Oper Status Speed MTU Type Remote Host Remote Port Summary ---------- ------------ ----------- ----- ----- -------- ---------------------------------- ----------- -------------------------------------------------------- bond998 up down 9216 bond bond999 up down 9216 bond br_default up up 9216 bridge IPv6 Address: fe80::1234:5678:9abc:def0/64 eth0 up up 100M 1500 eth mgmt-switch Eth105/1/17 IPv4 Address: 192.0.2.209/24 Address type: primary IPv6 Address: 2001:db8:1::495e/64 IPv6 Address: fe80::1234:5678:9abc:def0/64 lo up unknown 65536 loopback IPv4 Address: 127.0.0.1/8 Address type: primary IPv6 Address: ::1/128 mgmt up up 65575 vrf IPv4 Address: 127.0.0.1/8 IPv4 Address: 127.0.1.1/8 Address type: primary Address type: secondary IPv6 Address: ::1/128 swp1 up up 100G 9216 swp node1 e3b swp2 up up 100G 9216 swp node2 e3b swp3 up down 9216 swp swp4 up down 9216 swp swp5 up down 9216 swp swp6 up up 100G 9216 swp 952347XXXXXX e1b swp7 up down 9216 swp swp8 up down 9216 swp swp9 up down 9216 swp swp10 up down 9216 swp swp11 up down 9216 swp swp12 up down 9216 swp swp13 up down 9216 swp swp14 up down 9216 swp swp15 up down 9216 swp swp16 up down 9216 swp -
Verify information for the transceiver in the interface:
Cumulus Linux 5.4nv show interface --view=pluggablesShow example
cumulus@cumulus:mgmt:~$ nv show interface --view=pluggables Interface Identifier Vendor Name Vendor PN Vendor SN Vendor Rev --------- ------------- ----------- --------------- -------------- ---------- swp3 0x11 (QSFP28) Amphenol 112-00574 APF20379253516 B0 swp4 0x11 (QSFP28) AVAGO 332-00440 AF1815GU05Z A0 swp15 0x11 (QSFP28) Amphenol 112-00573 APF21109348001 B0 swp16 0x11 (QSFP28) Amphenol 112-00573 APF21109347895 B0
Cumulus Linux 5.11 and laternv show platform transceiverShow example
cumulus@cumulus:mgmt:~$ nv show platform transceiver Transceiver Identifier Vendor name Vendor PN Vendor SN Vendor revision ----------- ---------- ------------ --------- -------------- --------------- swp1 QSFP28 FINISAR CORP 332-00440 XA7C325 A1 swp2 QSFP28 FINISAR CORP 332-00440 XA7C33W A1 swp5 QSFP28 FINISAR CORP 332-00440 XA7C1RV A1 swp6 QSFP28 FINISAR CORP 332-00440 X4QANNS A1
-
Verify that the nodes each have a connection to each switch:
Cumulus Linux 5.4nv show interface --view=lldpShow example
cumulus@cumulus:mgmt:~$ net show lldp LocalPort Speed Mode RemoteHost RemotePort --------- ----- ---------- ---------------------- ----------- swp3 100G Trunk/L2 sw1 e3a swp4 100G Trunk/L2 sw2 e3b swp15 100G Trunk/L2 sw13 swp15 swp16 100G Trunk/L2 sw14 swp16
Cumulus Linux 5.11 and laternv show interface lldpShow example
cumulus@cumulus:mgmt:~$ nv show interface lldp Interface Speed Type Remote Host Remote Port --------- ----- ---- ---------------------------------- ----------- eth0 100M eth mgmt-switch Eth105/1/15 swp1 100G swp cluster1-01 e3a swp2 100G swp cluster1-02 e3a swp5 100G swp 952347XXXXXX e1a swp6 100G swp 952347XXXXXX e1a
-
Verify the health of storage ports and switch connectivity.
-
Verify the status of node storage ports:
storage port show -port-type ENETShow example
cluster1::*> storage port show -port-type ENET Speed Node Port Type Mode (Gb/s) State Status ------------------ ---- ----- ------- ------ -------- --------------- node1 e3a ENET storage 100 enabled online e3b ENET storage 100 enabled online node2 e3a ENET storage 100 enabled online e3b ENET storage 100 enabled online 4 entries were displayed.
-
-
Verify the shelf ports in the storage system:
storage shelf port show -fields remote-device, remote-portShow example
cluster1::*> storage shelf port show -fields remote-device, remote-port shelf id remote-port remote-device ----- -- ----------- ------------- 3.20 0 swp3 sw1 3.20 1 swp3 sw2 3.20 2 swp4 sw1 3.20 3 swp4 sw2 3.30 0 swp5 sw1 3.30 1 swp5 sw2 3.30 2 swp6 sw1 3.30 3 swp6 sw2 cluster1::*>
-
Verify that the switches are available:
network device-discovery show -protocol lldpShow example
cluster1::*> network device-discovery show -protocol lldp Node/ Local Discovered Protocol Port Device (LLDP: ChassisID) Interface Platform ----------- ------ ------------------------- ---------------- ---------------- node1 e0M mgmt-switch Eth123/1/41 - e1a data-switch Ethernet1/25 - e1b data-switch Ethernet1/26 - sw1 storage-switch swp1 - sw2 storage-switch swp1 - cs1 cluster1-01 swp3 - cs2 cluster2-02 swp3 - node2 e0M mgmt-switch Eth123/1/42 - e1a data-switch Ethernet1/27 - e1b data-switch Ethernet1/28 - sw1 storage-switch swp2 - sw2 storage-switch swp2 - cs1 cluster1-01 swp4 - cs2 cluster2-02 swp4 - 18 entries were displayed. -
Verify that the switches are being monitored.
system switch ethernet show -is-monitoring-enabled-operational trueShow example
cluster1::*> system switch ethernet show -is-monitoring-enabled-operational true Switch Type Address Model --------------------------- ------------------ ---------------- ----- sw1 storage-network 192.0.2.71 MSN2100-CB2RC Serial Number: MNXXXXXXGD Is Monitored: true Reason: None Software Version: Cumulus Linux version 5.11.1 running on Mellanox Technologies Ltd. MSN2100 Version Source: LLDP sw2 storage-network 192.0.2.72 MSN2100-CB2RC Serial Number: MNCXXXXXXGS Is Monitored: true Reason: None Software Version: Cumulus Linux version 5.11.1 running on Mellanox Technologies Ltd. MSN2100 Version Source: LLDP -
Repeat these verification steps on the partner (or other) switch.
After you've installed or upgraded the RCF, you can install the CSHM file.