Deploy BeeGFS clients

01/31/2023 Contributors

Optionally, Ansible can be used to configure BeeGFS clients and mount the file system.

Overview

Accessing BeeGFS file systems requires installing and configuring the BeeGFS client on each node that needs to mount the file system. This section documents how to perform these tasks using the available Ansible role.

Steps

Create the Client Inventory File

If needed, set up passwordless SSH from the Ansible control node to each of the hosts you want to configure as BeeGFS clients:
```
ssh-copy-id <user>@<HOSTNAME_OR_IP>
```
Under host_vars/, create a file for each BeeGFS client named <HOSTNAME>.yml with the following content, filling in the placeholder text with the correct information for your environment:
```
# BeeGFS Client
ansible_host: <MANAGEMENT_IP>
```

Optionally include one of the following if you want to use the NetApp E-Series Host Collection's roles to configure InfiniBand or Ethernet interfaces for clients to connect to BeeGFS file nodes:

If the network type is InfiniBand (using IPoIB):

eseries_ipoib_interfaces:
- name: <INTERFACE>  # Example: ib0 or i1b
  address: <IP/SUBNET> # Example: 100.127.100.1/16
- name: <INTERFACE>  # Additional interfaces as needed.
  address: <IP/SUBNET>

If the network type is RDMA over Converged Ethernet (RoCE):

eseries_roce_interfaces:
- name: <INTERFACE>  # Example: eth0.
  address: <IP/SUBNET> # Example: 100.127.100.1/16
- name: <INTERFACE>  # Additional interfaces as needed.
  address: <IP/SUBNET>

If the network type is Ethernet (TCP only, no RDMA):

eseries_ip_interfaces:
- name: <INTERFACE>  # Example: eth0.
  address: <IP/SUBNET> # Example: 100.127.100.1/16
- name: <INTERFACE>  # Additional interfaces as needed.
  address: <IP/SUBNET>

Create a new file client_inventory.yml and specify the user Ansible should use to connect to each client, and the password Ansible should use for privilege escalation (this requires ansible_ssh_user be root, or have sudo privileges):

# BeeGFS client inventory.
all:
  vars:
    ansible_ssh_user: <USER>
    ansible_become_password: <PASSWORD>

Do not store passwords in plain text. Instead, use the Ansible Vault (see the Ansible documentation for Encrypting content with Ansible Vault) or use the --ask-become-pass option when running the playbook.

In the client_inventory.yml file, list all hosts that should be configured as BeeGFS clients under the beegfs_clients group, and then refer to the inline comments and uncomment any additional configuration required to build the BeeGFS client kernel module on your system:

children:
    # Ansible group representing all BeeGFS clients:
    beegfs_clients:
      hosts:
        <CLIENT HOSTNAME>:
        # Additional clients as needed.

      vars:
        # OPTION 1: If you’re using the Mellanox OFED drivers and they are already installed:
        #eseries_ib_skip: True # Skip installing inbox drivers when using the IPoIB role.
        #beegfs_client_ofed_enable: True
        #beegfs_client_ofed_include_path: "/usr/src/ofa_kernel/default/include"

        # OPTION 2: If you’re using inbox IB/RDMA drivers and they are already installed:
        #eseries_ib_skip: True # Skip installing inbox drivers when using the IPoIB role.

        # OPTION 3: If you want to use inbox IB/RDMA drivers and need them installed/configured.
        #eseries_ib_skip: False # Default value.
        #beegfs_client_ofed_enable: False # Default value.

When using the Mellanox OFED drivers, make sure that beegfs_client_ofed_include_path points to the correct "header include path" for your Linux installation. For more information, see the BeeGFS documentation for RDMA support.

In the client_inventory.yml file, list the BeeGFS file systems you want mounted under any previously defined vars:

        beegfs_client_mounts:
          - sysMgmtdHost: <IP ADDRESS>  # Primary IP of the BeeGFS management service.
            mount_point: /mnt/beegfs    # Path to mount BeeGFS on the client.
            connInterfaces:
              - <INTERFACE> # Example: ibs4f1
              - <INTERFACE>
            beegfs_client_config:
              # Maximum number of simultaneous connections to the same node.
              connMaxInternodeNum: 128 # BeeGFS Client Default: 12
              # Allocates the number of buffers for transferring IO.
              connRDMABufNum: 36 # BeeGFS Client Default: 70
              # Size of each allocated RDMA buffer
              connRDMABufSize: 65536 # BeeGFS Client Default: 8192
              # Required when using the BeeGFS client with the shared-disk HA solution.
              # This does require BeeGFS targets be mounted in the default “sync” mode.
              # See the documentation included with the BeeGFS client role for full details.
              sysSessionChecksEnabled: false
        # Specify additional file system mounts for this or other file systems.

As of BeeGFS 7.2.7 and 7.3.1 connection authentication must be configured or explicitly disabled. Depending how you choose to configure connection based authentication when specifying common file node configuration, you may need to adjust your client configuration:
1. By default the HA cluster deployment will automatically configure connection authentication, and generate a connauthfile that will be placed/maintained on the Ansible control node at <INVENTORY>/files/beegfs/<sysMgmtdHost>_connAuthFile. By default the BeeGFS client role is setup to read/distribute this file to the clients defined in client_inventory.yml, and no additional action is needed.
  1. For advanced options refer to the full list of defaults included with the BeeGFS client role.
2. If you choose to specify a custom secret with beegfs_ha_conn_auth_secret specify it in the client_inventory.yml file as well:
  beegfs_ha_conn_auth_secret: <SECRET>
3. If you choose to disable connection based authentication entirely with beegfs_ha_conn_auth_enabled, specify that in the client_inventory.yml file as well:
  beegfs_ha_conn_auth_enabled: false

For a full list of supported parameters and additional details refer to the full BeeGFS client documentation. For a complete example of a client inventory click here.

Create the BeeGFS Client Playbook File

Create a new file client_playbook.yml

# BeeGFS client playbook.
- hosts: beegfs_clients
  any_errors_fatal: true
  gather_facts: true
  collections:
    - netapp_eseries.beegfs
    - netapp_eseries.host
  tasks:

Optional: If you want to use the NetApp E-Series Host Collection's roles to configure interfaces for clients to connect to BeeGFS file systems, import the role corresponding with the interface type you are configuring:
1. If you are using are using InfiniBand (IPoIB):
  - name: Ensure IPoIB is configured import_role: name: ipoib
2. If you are using are using RDMA over Converged Ethernet (RoCE):
  - name: Ensure IPoIB is configured import_role: name: roce
3. If you are using are using Ethernet (TCP only, no RDMA):
  - name: Ensure IPoIB is configured import_role: name: ip

Lastly import the BeeGFS client role to install the client software and setup the file system mounts:

    # REQUIRED: Install the BeeGFS client and mount the BeeGFS file system.
    - name: Verify the BeeGFS clients are configured.
      import_role:
        name: beegfs_client

For a complete example of a client playbook click here.

Run the BeeGFS Client Playbook

To install/build the client and mount BeeGFS, run the following command:

ansible-playbook -i client_inventory.yml client_playbook.yml