English

Configure NetApp H615c (Manual Deployment)

Contributors netapp-dorianh Download PDF of this page

In this solution, the NetApp H615c compute nodes are configured as Kubernetes worker nodes. The Inferencing workload is hosted on these nodes.

Deploying the compute nodes involves the following tasks:

  • Install Ubuntu 18.04.4 LTS.

  • Configure networking for data and management access.

  • Prepare the Ubuntu instances for Kubernetes deployment.

Install Ubuntu 18.04.4 LTS

The following high-level steps are required to install the operating system on the H615c compute nodes:

  1. Download Ubuntu 18.04.4 LTS from Ubuntu releases.

  2. Using a browser, connect to the IPMI of the H615c node and launch Remote Control.

  3. Map the Ubuntu ISO using the Virtual Media Wizard and start the installation.

  4. Select one of the two physical interfaces as the Primary network interface when prompted.

    An IP from a DHCP source is allocated when available, or you can switch to a manual IP configuration later. The network configuration is modified to a bond-based setup after the OS has been installed.

  5. Provide a hostname followed by a domain name.

  6. Create a user and provide a password.

  7. Partition the disks according to your requirements.

  8. Under Software Selection, select OpenSSH server and click Continue.

  9. Reboot the node.

Configure Networking for Data and Management Access

The two physical network interfaces of the Kubernetes worker nodes are set up as a bond and VLAN interfaces for management and application, and NFS data traffic is created on top of it.

The inferencing applications and associated containers use the application network for connectivity.
  1. Connect to the console of the Ubuntu instance as a user with root privileges and launch a terminal session.

  2. Navigate to /etc/netplan and open the 01-netcfg.yaml file.

  3. Update the netplan file based on the network details for the management, application, and NFS traffic in your environment.

    The following template of the netplan file was used in this solution:

    # This file describes the network interfaces available on your system
    # For more information, see netplan(5).
    network:
      version: 2
      renderer: networkd
      ethernets:
        enp59s0f0: #Physical Interface 1
          match:
            macaddress: <<mac_address Physical Interface 1>>
          set-name: enp59s0f0
          mtu: 9000
        enp59s0f1: # Physical Interface 2
          match:
            macaddress: <<mac_address Physical Interface 2>>
          set-name: enp59s0f1
          mtu: 9000
      bonds:
          bond0:
            mtu: 9000
            dhcp4: false
            dhcp6: false
            interfaces: [ enp59s0f0, enp59s0f1 ]
            parameters:
              mode: 802.3ad
              mii-monitor-interval: 100
      vlans:
        vlan.3488: #Management VLAN
          id: 3488
          link: bond0
          dhcp4: false
          addresses: [ipv4_address/subnet]
          routes:
          - to: 0.0.0.0/0
            via: 172.21.232.111
            metric: 100
            table: 3488
          - to: x.x.x.x/x # Additional routes if any
            via: y.y.y.y
            metric: <<metric>>
            table: <<table #>>
          routing-policy:
          - from: 0.0.0.0/0
            priority: 32768#Higher Priority than table 3487
            table: 3488
          nameservers:
            addresses: [nameserver_ip]
            search: [ search_domain ]
          mtu: 1500
        vlan.3487:
          id: 3487
          link: bond0
          dhcp4: false
          addresses: [ipv4_address/subnet]
          routes:
          - to: 0.0.0.0/0
            via: 172.21.231.111
            metric: 101
            table: 3487
          - to: x.x.x.x/x
            via: y.y.y.y
            metric: <<metric>>
            table: <<table #>>
          routing-policy:
          - from: 0.0.0.0/0
            priority: 32769#Lower Priority
            table: 3487
          nameservers:
            addresses: [nameserver_ip]
            search: [ search_domain ]
          mtu: 1500    vlan.3491:
          id: 3491
          link: bond0
          dhcp4: false
          addresses: [ipv4_address/subnet]
          mtu: 9000
  4. Confirm that the priorities for the routing policies are lower than the priorities for the main and default tables.

  5. Apply the netplan.

    sudo netplan -–debug apply
  6. Make sure that there are no errors.

  7. If Network Manager is running, stop and disable it.

    systemctl stop NetworkManager
    systemctl disable NetworkManager
  8. Add a host record for the server in DNS.

  9. Open a VI editor to /etc/iproute2/rt_tables and add the two entries.

    #
    # reserved values
    #
    255     local
    254     main
    253     default
    0       unspec
    #
    # local
    #
    #1      inr.ruhep
    101     3488
    102     3487
  10. Match the table number to what you used in the netplan.

  11. Open a VI editor to /etc/sysctl.conf and set the value of the following parameters.

    net.ipv4.conf.default.rp_filter=0
    net.ipv4.conf.all.rp_filter=0net.ipv4.ip_forward=1
  12. Update the system.

    sudo apt-get update && sudo apt-get upgrade
  13. Reboot the system

  14. Repeat steps 1 through 13 for the other Ubuntu instance.