Skip to main content
Cluster and storage switches

Install Cumulus Linux in Cumulus mode

Contributors netapp-yvonneo
PDFs

Follow this procedure to install Cumulus Linux (CL) OS when the switch is running in Cumulus mode.

Note Cumulus Linux (CL) OS can be installed either when the switch is running Cumulus Linux or ONIE (see Install in ONIE mode).
What you'll need

  • Intermediate-level Linux knowledge.

  • Familiarity with basic text editing, UNIX file permissions, and process monitoring. A variety of text editors are pre-installed, including vi and nano.

  • Access to a Linux or UNIX shell. If you are running Windows, use a Linux environment as your command line tool for interacting with Cumulus Linux.

  • The baud rate requirement is set to 115200 on the serial console switch for NVIDIA SN2100 switch console access, as follows:

    • 115200 baud

    • 8 data bits

    • 1 stop bit

    • parity: none

    • flow control: none

About this task

Be aware of the following:

Note Each time Cumulus Linux is installed, the entire file system structure is erased and rebuilt.
Caution The default password for the cumulus user account is cumulus. The first time you log into Cumulus Linux, you must change this default password. Be sure to update any automation scripts before installing a new image. Cumulus Linux provides command line options to change the default password automatically during the installation process.
Example 1. Steps
Cumulus Linux 4.4.3

  1. Log in to the switch.

    First time log in to the switch requires username/password of cumulus/cumulus with sudo privileges.

    cumulus login: cumulus
Password: cumulus
You are required to change your password immediately (administrator enforced)
Changing password for cumulus.
Current password: cumulus
New password: <new_password>
Retype new password: <new_password>

  2. Check the Cumulus Linux version: net show system

    cumulus@cumulus:mgmt:~$ net show system
Hostname......... cumulus
Build............ Cumulus Linux 4.4.3
Uptime........... 0:08:20.860000
Model............ Mlnx X86
CPU.............. x86_64 Intel Atom C2558 2.40GHz
Memory........... 8GB
Disk............. 14.7GB
ASIC............. Mellanox Spectrum MT52132
Ports............ 16 x 100G-QSFP28
Part Number...... MSN2100-CB2FC
Serial Number.... MT2105T05177
Platform Name.... x86_64-mlnx_x86-r0
Product Name..... MSN2100
ONIE Version..... 2019.11-5.2.0020-115200
Base MAC Address. 04:3F:72:43:92:80
Manufacturer..... Mellanox

  3. Configure the hostname, IP address, subnet mask, and default gateway. The new hostname only becomes effective after restarting the console/SSH session.

    Note A Cumulus Linux switch provides at least one dedicated Ethernet management port called eth0. This interface is specifically for out-of-band management use. By default, the management interface uses DHCPv4 for addressing.
    Caution Do not use an underscore (_), apostrophe ('), or non-ASCII characters in the hostname. 
    cumulus@cumulus:mgmt:~$ net add hostname sw1
cumulus@cumulus:mgmt:~$ net add interface eth0 ip address 10.233.204.71
cumulus@cumulus:mgmt:~$ net add interface eth0 ip gateway 10.233.204.1
cumulus@cumulus:mgmt:~$ net pending
cumulus@cumulus:mgmt:~$ net commit

    This command modifies both the /etc/hostname and /etc/hosts files.

  4. Confirm that the hostname, IP address, subnet mask, and default gateway have been updated.

    cumulus@sw1:mgmt:~$ hostname sw1
cumulus@sw1:mgmt:~$ ifconfig eth0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
inet 10.233.204.71  netmask 255.255.254.0  broadcast 10.233.205.255
inet6 fe80::bace:f6ff:fe19:1df6  prefixlen 64  scopeid 0x20<link>
ether b8:ce:f6:19:1d:f6  txqueuelen 1000  (Ethernet)
RX packets 75364  bytes 23013528 (21.9 MiB)
RX errors 0  dropped 7  overruns 0  frame 0
TX packets 4053  bytes 827280 (807.8 KiB)
TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0 device memory 0xdfc00000-dfc1ffff

cumulus@sw1::mgmt:~$ ip route show vrf mgmt
default via 10.233.204.1 dev eth0
unreachable default metric 4278198272
10.233.204.0/23 dev eth0 proto kernel scope link src 10.233.204.71
127.0.0.0/8 dev mgmt proto kernel scope link src 127.0.0.1

  5. Set the date, time, time zone, and NTP server on the switch.

    1. Verify the current time zone:

      cumulus@sw1:~$ cat /etc/timezone

    2. Update to the new time zone:

      cumulus@sw1:~$ sudo dpkg-reconfigure --frontend noninteractive tzdata

    3. Verify your current time zone:

      cumulus@switch:~$ date +%Z

    4. To set the time zone using the guided wizard, run the following command:

      cumulus@sw1:~$ sudo dpkg-reconfigure tzdata

    5. Set the software clock according to the configured time zone:

      cumulus@switch:~$ sudo date -s "Tue Oct 28 00:37:13 2023"

    6. Set the current value of the software clock to the hardware clock:

      cumulus@switch:~$ sudo hwclock -w

    7. Add an NTP server if required:

      cumulus@sw1:~$ net add time ntp server <cumulus.network.ntp.org> iburst
cumulus@sw1:~$ net pending
cumulus@sw1:~$ net commit

    8. Verify that ntpd is running on the system:

      cumulus@sw1:~$ ps -ef | grep ntp
ntp       4074     1  0 Jun20 ?        00:00:33 /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 101:102

    9. Specify the NTP source interface. By default, the source interface that NTP uses is eth0. You can configure a different NTP source interface as follows:

      cumulus@sw1:~$ net add time ntp source <src_int>
cumulus@sw1:~$ net pending
cumulus@sw1:~$ net commit

  6. Install Cumulus Linux 4.4.3:

    cumulus@sw1:mgmt:~$ sudo onie-install -a -i http://<web-server>/<path>/cumulus-linux-4.4.3-mlx-amd64.bin

    The installer starts the download. Type y when prompted.

  7. Reboot the NVIDIA SN2100 switch:

    cumulus@sw1:mgmt:~$ sudo reboot

  8. The installation starts automatically, and the following GRUB screen choices appear. Do not make any selections.

    • Cumulus-Linux GNU/Linux

    • ONIE: Install OS

    • CUMULUS-INSTALL

    • Cumulus-Linux GNU/Linux

  9. Repeat steps 1 to 4 to log in.

  10. Verify that the Cumulus Linux version is 4.4.3: net show version

    cumulus@sw1:mgmt:~$ net show version
NCLU_VERSION=1.0-cl4.4.3u0
DISTRIB_ID="Cumulus Linux"
DISTRIB_RELEASE=4.4.3
DISTRIB_DESCRIPTION="Cumulus Linux 4.4.3"

  11. Create a new user and add this user to the sudo group. This user only becomes effective after the console/SSH session is restarted.

    sudo adduser --ingroup netedit admin

    cumulus@sw1:mgmt:~$ sudo adduser --ingroup netedit admin
[sudo] password for cumulus:
Adding user 'admin' ...
Adding new user 'admin' (1001) with group `netedit' ...
Creating home directory '/home/admin' ...
Copying files from '/etc/skel' ...
New password:
Retype new password:
passwd: password updated successfully
Changing the user information for admin
Enter the new value, or press ENTER for the default
Full Name []:
Room Number []:
Work Phone []:
Home Phone []:
Other []:
Is the information correct? [Y/n] y

cumulus@sw1:mgmt:~$ sudo adduser admin sudo
[sudo] password for cumulus:
Adding user `admin' to group `sudo' ...
Adding user admin to group sudo
Done.
cumulus@sw1:mgmt:~$ exit
logout
Connection to 10.233.204.71 closed.

[admin@cycrh6svl01 ~]$ ssh admin@10.233.204.71
admin@10.233.204.71's password:
Linux sw1 4.19.0-cl-1-amd64 #1 SMP Cumulus 4.19.206-1+cl4.4.1u1 (2021-09-09) x86_64
Welcome to NVIDIA Cumulus (R) Linux (R)

For support and online technical documentation, visit
http://www.cumulusnetworks.com/support

The registered trademark Linux (R) is used pursuant to a sublicense from LMI, the exclusive licensee of Linus Torvalds, owner of the mark on a world-wide basis.
admin@sw1:mgmt:~$
Cumulus Linux 5.x

  1. Log in to the switch.

    First time log in to the switch requires username/password of cumulus/cumulus with sudo privileges.

    cumulus login: cumulus
Password: cumulus
You are required to change your password immediately (administrator enforced)
Changing password for cumulus.
Current password: cumulus
New password: <new_password>
Retype new password: <new_password>

  2. Check the Cumulus Linux version: nv show system

    cumulus@cumulus:mgmt:~$ nv show system
operational         applied              description
------------------- -------------------- ---------------------
hostname            cumulus              cumulus
build               Cumulus Linux 5.3.0  system build version
uptime              6 days, 8:37:36      system uptime
timezone            Etc/UTC              system time zone

  3. Configure the hostname, IP address, subnet mask, and default gateway. The new hostname only becomes effective after restarting the console/SSH session.

    Note A Cumulus Linux switch provides at least one dedicated Ethernet management port called eth0. This interface is specifically for out-of-band management use. By default, the management interface uses DHCPv4 for addressing.
    Caution Do not use an underscore (_), apostrophe ('), or non-ASCII characters in the hostname. 
    cumulus@cumulus:mgmt:~$ nv set system hostname sw1
cumulus@cumulus:mgmt:~$ nv set interface eth0 ip address 10.233.204.71/24
cumulus@cumulus:mgmt:~$ nv set interface eth0 ip gateway 10.233.204.1
cumulus@cumulus:mgmt:~$ nv config apply
cumulus@cumulus:mgmt:~$ nv config save

    This command modifies both the /etc/hostname and /etc/hosts files.

  4. Confirm that the hostname, IP address, subnet mask, and default gateway have been updated.

    cumulus@sw1:mgmt:~$ hostname sw1
cumulus@sw1:mgmt:~$ ifconfig eth0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
inet 10.233.204.71  netmask 255.255.254.0  broadcast 10.233.205.255
inet6 fe80::bace:f6ff:fe19:1df6  prefixlen 64  scopeid 0x20<link>
ether b8:ce:f6:19:1d:f6  txqueuelen 1000  (Ethernet)
RX packets 75364  bytes 23013528 (21.9 MiB)
RX errors 0  dropped 7  overruns 0  frame 0
TX packets 4053  bytes 827280 (807.8 KiB)
TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0 device memory 0xdfc00000-dfc1ffff

cumulus@sw1::mgmt:~$ ip route show vrf mgmt
default via 10.233.204.1 dev eth0
unreachable default metric 4278198272
10.233.204.0/23 dev eth0 proto kernel scope link src 10.233.204.71
127.0.0.0/8 dev mgmt proto kernel scope link src 127.0.0.1

  5. Set the time zone, date, time, and NTP server on the switch.

    1. Set the time zone:

      cumulus@sw1:~$ nv set system timezone US/Eastern
cumulus@sw1:~$ nv config apply

    2. Verify your current time zone:

      cumulus@switch:~$ date +%Z

    3. To set the time zone using the guided wizard, run the following command:

      cumulus@sw1:~$ sudo dpkg-reconfigure tzdata

    4. Set the software clock according to the configured time zone:

      cumulus@sw1:~$ sudo date -s "Tue Oct 28 00:37:13 2023"

    5. Set the current value of the software clock to the hardware clock:

      cumulus@sw1:~$ sudo hwclock -w

    6. Add an NTP server if required:

      cumulus@sw1:~$ nv set service ntp default server <cumulus.network.ntp.org> iburst on
cumulus@sw1:~$ nv config apply
cumulus@sw1:~$ nv config save

    7. Verify that ntpd is running on the system:

      cumulus@sw1:~$ ps -ef | grep ntp
ntp       4074     1  0 Jun20 ?        00:00:33 /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 101:102

    8. Specify the NTP source interface. By default, the source interface that NTP uses is eth0. You can configure a different NTP source interface as follows:

      cumulus@sw1:~$ nv set service ntp default listen <src_int>
cumulus@sw1:~$ nv config apply

  6. Install Cumulus Linux 5.4:

    cumulus@sw1:mgmt:~$ sudo onie-install -a -i http://<web-server>/<path>/cumulus-linux-5.4-mlx-amd64.bin

    The installer starts the download. Type y when prompted.

  7. Reboot the NVIDIA SN2100 switch:

    cumulus@sw1:mgmt:~$ sudo reboot

  8. The installation starts automatically, and the following GRUB screen choices appear. Do not make any selections.

    • Cumulus-Linux GNU/Linux

    • ONIE: Install OS

    • CUMULUS-INSTALL

    • Cumulus-Linux GNU/Linux

  9. Repeat steps 1 to 4 to log in.

  10. Verify that the Cumulus Linux version is 5.4: nv show system

    cumulus@cumulus:mgmt:~$ nv show system
operational         applied              description
------------------- -------------------- ---------------------
hostname            cumulus              cumulus
build               Cumulus Linux 5.4.0  system build version
uptime              6 days, 13:37:36     system uptime
timezone            Etc/UTC              system time zone

  11. Verify that the nodes each have a connection to each switch:

    cumulus@sw1:mgmt:~$ net show lldp

LocalPort  Speed  Mode        RemoteHost                          RemotePort
---------  -----  ----------  ----------------------------------  -----------
eth0       100M   Mgmt        mgmt-sw1                            Eth110/1/29
swp2s1     25G    Trunk/L2    node1                               e0a
swp15      100G   BondMember  sw2                                 swp15
swp16      100G   BondMember  sw2                                 swp16

  12. Create a new user and add this user to the sudo group. This user only becomes effective after the console/SSH session is restarted.

    sudo adduser --ingroup netedit admin

    cumulus@sw1:mgmt:~$ sudo adduser --ingroup netedit admin
[sudo] password for cumulus:
Adding user 'admin' ...
Adding new user 'admin' (1001) with group `netedit' ...
Creating home directory '/home/admin' ...
Copying files from '/etc/skel' ...
New password:
Retype new password:
passwd: password updated successfully
Changing the user information for admin
Enter the new value, or press ENTER for the default
Full Name []:
Room Number []:
Work Phone []:
Home Phone []:
Other []:
Is the information correct? [Y/n] y

cumulus@sw1:mgmt:~$ sudo adduser admin sudo
[sudo] password for cumulus:
Adding user `admin' to group `sudo' ...
Adding user admin to group sudo
Done.
cumulus@sw1:mgmt:~$ exit
logout
Connection to 10.233.204.71 closed.

[admin@cycrh6svl01 ~]$ ssh admin@10.233.204.71
admin@10.233.204.71's password:
Linux sw1 4.19.0-cl-1-amd64 #1 SMP Cumulus 4.19.206-1+cl4.4.1u1 (2021-09-09) x86_64
Welcome to NVIDIA Cumulus (R) Linux (R)

For support and online technical documentation, visit
http://www.cumulusnetworks.com/support

The registered trademark Linux (R) is used pursuant to a sublicense from LMI, the exclusive licensee of Linus Torvalds, owner of the mark on a world-wide basis.
admin@sw1:mgmt:~$

  13. Add additional user groups for the admin user to access nv commands:

    cumulus@sw1:mgmt:~$ sudo adduser admin nvshow
     [sudo] password for cumulus:
     Adding user 'admin' to group 'nvshow' ...
     Adding user admin to group nvshow
     Done.

    See NVIDIA User Accounts for more information.

What's next?

Install the Reference Configuration File (RCF) script.