Install the RCF script

02/22/2023 Contributors

PDFs

Follow this procedure to install the RCF script.

What you'll need

Before installing the RCF script, make sure that the following are available on the switch:

Cumulus Linux 4.4.3 is installed.
IP address, subnet mask, and default gateway defined via DHCP or manually configured.

Current RCF script versions

There are two RCF scripts available for Clustering and Storage applications. The procedure for each is the same.

Clustering: MSN2100-RCF-v1.8-Cluster
Storage: MSN2100-RCF-v1.8-Storage

The following example procedure shows how to download and apply the RCF script for Cluster switches.

Example command output uses switch management IP address 10.233.204.71, netmask 255.255.254.0 and default gateway 10.233.204.1.

Steps

Display the available interfaces on the SN2100 switch:

net show interface all

Show example

cumulus@cumulus:mgmt:~$ net show interface all

State  Name   Spd  MTU    Mode         LLDP                Summary
-----  -----  ---  -----  -----------  ------------------  --------------
...
...
ADMDN  swp1   N/A  9216   NotConfigured
ADMDN  swp2   N/A  9216   NotConfigured
ADMDN  swp3   N/A  9216   NotConfigured
ADMDN  swp4   N/A  9216   NotConfigured
ADMDN  swp5   N/A  9216   NotConfigured
ADMDN  swp6   N/A  9216   NotConfigured
ADMDN  swp7   N/A  9216   NotConfigure
ADMDN  swp8   N/A  9216   NotConfigured
ADMDN  swp9   N/A  9216   NotConfigured
ADMDN  swp10  N/A  9216   NotConfigured
ADMDN  swp11  N/A  9216   NotConfigured
ADMDN  swp12  N/A  9216   NotConfigured
ADMDN  swp13  N/A  9216   NotConfigured
ADMDN  swp14  N/A  9216   NotConfigured
ADMDN  swp15  N/A  9216   NotConfigured
ADMDN  swp16  N/A  9216   NotConfigured

Copy the RCF python script to the switch:

cumulus@cumulus:mgmt:~$ pwd
/home/cumulus
cumulus@cumulus:mgmt: /tmp$ scp <user>@<host:/<path>/MSN2100-RCF-v1.8-Cluster
ssologin@10.233.204.71's password:
MSN2100-RCF-v1.8-Cluster                 100% 8607   111.2KB/s         00:00

Apply the RCF python script MSN2100-RCF-v1.8-Cluster:

cumulus@cumulus:mgmt:/tmp$ sudo python3 MSN2100-RCF-v1.8-Cluster
[sudo] password for cumulus:
...
Step 1: Creating the banner file
Step 2: Registering banner message
Step 3: Updating the MOTD file
Step 4: Ensuring passwordless use of cl-support command by admin
Step 5: Disabling apt-get
Step 6: Creating the interfaces
Step 7: Adding the interface config
Step 8: Disabling cdp
Step 9: Adding the lldp config
Step 10: Adding the RoCE base config
Step 11: Modifying RoCE Config
Step 12: Configure SNMP
Step 13: Reboot the switch

The RCF script completes the steps listed above.

For any RCF python script issues that cannot be corrected, contact NetApp Support for assistance.

Verify the configuration after the reboot:

net show interface all

Show example

cumulus@cumulus:mgmt:~$ net show interface all

State  Name      Spd   MTU    Mode       LLDP              Summary
-----  --------- ----  -----  ---------- ----------------- --------
...
...
DN     swp1s0    N/A   9216   Trunk/L2                     Master: bridge(UP)
DN     swp1s1    N/A   9216   Trunk/L2                     Master: bridge(UP)
DN     swp1s2    N/A   9216   Trunk/L2                     Master: bridge(UP)
DN     swp1s3    N/A   9216   Trunk/L2                     Master: bridge(UP)
DN     swp2s0    N/A   9216   Trunk/L2                     Master: bridge(UP)
DN     swp2s1    N/A   9216   Trunk/L2                     Master: bridge(UP)
DN     swp2s2    N/A   9216   Trunk/L2                     Master: bridge(UP)
DN     swp2s3    N/A   9216   Trunk/L2                     Master: bridge(UP)
UP     swp3      100G  9216   Trunk/L2                     Master: bridge(UP)
UP     swp4      100G  9216   Trunk/L2                     Master: bridge(UP)
DN     swp5      N/A   9216   Trunk/L2                     Master: bridge(UP)
DN     swp6      N/A   9216   Trunk/L2                     Master: bridge(UP)
DN     swp7      N/A   9216   Trunk/L2                     Master: bridge(UP)
DN     swp8      N/A   9216   Trunk/L2                     Master: bridge(UP)
DN     swp9      N/A   9216   Trunk/L2                     Master: bridge(UP)
DN     swp10     N/A   9216   Trunk/L2                     Master: bridge(UP)
DN     swp11     N/A   9216   Trunk/L2                     Master: bridge(UP)
DN     swp12     N/A   9216   Trunk/L2                     Master: bridge(UP)
DN     swp13     N/A   9216   Trunk/L2                     Master: bridge(UP)
DN     swp14     N/A   9216   Trunk/L2                     Master: bridge(UP)
UP     swp15     N/A   9216   BondMember                   Master: bond_15_16(UP)
UP     swp16     N/A   9216   BondMember                   Master: bond_15_16(UP)
...
...

cumulus@cumulus:mgmt:~$ net show roce config
RoCE mode.......... lossless
Congestion Control:
  Enabled SPs.... 0 2 5
  Mode........... ECN
  Min Threshold.. 150 KB
  Max Threshold.. 1500 KB
PFC:
  Status......... enabled
  Enabled SPs.... 2 5
  Interfaces......... swp10-16,swp1s0-3,swp2s0-3,swp3-9

DSCP                     802.1p  switch-priority
-----------------------  ------  ---------------
0 1 2 3 4 5 6 7               0                0
8 9 10 11 12 13 14 15         1                1
16 17 18 19 20 21 22 23       2                2
24 25 26 27 28 29 30 31       3                3
32 33 34 35 36 37 38 39       4                4
40 41 42 43 44 45 46 47       5                5
48 49 50 51 52 53 54 55       6                6
56 57 58 59 60 61 62 63       7                7

switch-priority  TC  ETS
---------------  --  --------
0 1 3 4 6 7       0  DWRR 28%
2                 2  DWRR 28%
5                 5  DWRR 43%

Verify information for the transceiver in the interface:

net show interface pluggables

Show example

cumulus@cumulus:mgmt:~$ net show interface pluggables
Interface  Identifier     Vendor Name  Vendor PN        Vendor SN       Vendor Rev
---------  -------------  -----------  ---------------  --------------  ----------
swp3       0x11 (QSFP28)  Amphenol     112-00574        APF20379253516  B0
swp4       0x11 (QSFP28)  AVAGO        332-00440        AF1815GU05Z     A0
swp15      0x11 (QSFP28)  Amphenol     112-00573        APF21109348001  B0
swp16      0x11 (QSFP28)  Amphenol     112-00573        APF21109347895  B0

Verify that the nodes each have a connection to each switch:

net show lldp

Show example

cumulus@cumulus:mgmt:~$ net show lldp

LocalPort  Speed  Mode        RemoteHost              RemotePort
---------  -----  ----------  ----------------------  -----------
swp3       100G   Trunk/L2    sw1                     e3a
swp4       100G   Trunk/L2    sw2                     e3b
swp15      100G   BondMember  sw13                    swp15
swp16      100G   BondMember  sw14                    swp16

Verify the health of cluster ports on the cluster.

Verify that e0d ports are up and healthy across all nodes in the cluster:

network port show -role cluster

Show example

cluster1::*> network port show -role cluster

Node: node1
                                                                       Ignore
                                                  Speed(Mbps) Health   Health
Port      IPspace      Broadcast Domain Link MTU  Admin/Oper  Status   Status
--------- ------------ ---------------- ---- ---- ----------- -------- ------
e3a       Cluster      Cluster          up   9000  auto/10000 healthy  false
e3b       Cluster      Cluster          up   9000  auto/10000 healthy  false

Node: node2
                                                                       Ignore
                                                  Speed(Mbps) Health   Health
Port      IPspace      Broadcast Domain Link MTU  Admin/Oper  Status   Status
--------- ------------ ---------------- ---- ---- ----------- -------- ------
e3a       Cluster      Cluster          up   9000  auto/10000 healthy  false
e3b       Cluster      Cluster          up   9000  auto/10000 healthy  false

Verify the switch health from the cluster (this might not show switch sw2, since LIFs are not homed on e0d).

Show example

cluster1::*> network device-discovery show -protocol lldp
Node/       Local  Discovered
Protocol    Port   Device (LLDP: ChassisID)  Interface Platform
----------- ------ ------------------------- --------- ----------
node1/lldp
            e3a    sw1 (b8:ce:f6:19:1a:7e)   swp3      -
            e3b    sw2 (b8:ce:f6:19:1b:96)   swp3      -

node2/lldp
            e3a    sw1 (b8:ce:f6:19:1a:7e)   swp4      -
            e3b    sw2 (b8:ce:f6:19:1b:96)   swp4      -


cluster1::*> system switch ethernet show -is-monitoring-enabled-operational true
Switch                      Type               Address          Model
--------------------------- ------------------ ---------------- -----
sw1                         cluster-network    10.233.205.90    MSN2100-CB2RC
     Serial Number: MNXXXXXXGD
      Is Monitored: true
            Reason: None
  Software Version: Cumulus Linux version 4.4.3 running on Mellanox
                    Technologies Ltd. MSN2100
    Version Source: LLDP

sw2                         cluster-network    10.233.205.91    MSN2100-CB2RC
     Serial Number: MNCXXXXXXGS
      Is Monitored: true
            Reason: None
  Software Version: Cumulus Linux version 4.4.3 running on Mellanox
                    Technologies Ltd. MSN2100
    Version Source: LLDP

What's next?

Configure switch log collection.

Install the RCF script

Creating your file...