Upgrade to BeeGFS v8

01/27/2026 Contributors

Follow these steps to upgrade your BeeGFS HA cluster from version 7.4.6 to BeeGFS v8.

Overview

BeeGFS v8 introduces several significant changes which require additional setup before upgrading from BeeGFS v7. This document guides you through preparing your cluster for BeeGFS v8's new requirements, and then upgrading to BeeGFS v8.

Before upgrading to BeeGFS v8, ensure your system is running at least BeeGFS 7.4.6. Any cluster running a release previous to BeeGFS 7.4.6 must first upgrade to version 7.4.6 before proceeding with this BeeGFS v8 upgrade procedure.

Key changes in BeeGFS v8

BeeGFS v8 introduces the following major changes:

License enforcement: BeeGFS v8 requires a license to use premium features such as storage pools, remote storage targets, BeeOND, and more. Acquire a valid license for your BeeGFS cluster before upgrading. If needed, you can obtain a temporary BeeGFS v8 evaluation license from the BeeGFS License Portal.
Management Service Database Migration: To enable configuration with the new TOML-based format in BeeGFS v8, you must manually migrate your BeeGFS v7 management service database to the updated BeeGFS v8 format.
TLS encryption: BeeGFS v8 introduces TLS for secure communication between services. You will need to generate and distribute TLS certificates for the BeeGFS management service and the beegfs command-line utility as part of the upgrade.

For more details and additional changes in BeeGFS 8, see the BeeGFS v8.0.0 Upgrade Guide.

Upgrading to BeeGFS v8 requires cluster downtime. In addition, BeeGFS v7 clients cannot connect to BeeGFS v8 clusters. Coordinate the upgrade timing between the cluster and clients carefully to minimize impact on operations.

Prepare your BeeGFS cluster for the upgrade

Before starting the upgrade, carefully prepare your environment to ensure a smooth transition and minimize downtime.

Ensure your cluster is in a healthy state, with all BeeGFS services running on their preferred nodes. From a file node running BeeGFS services, verify all Pacemaker resources are running on their preferred nodes:
```
pcs status
```

Record and back up your cluster configuration.

Refer to the BeeGFS Backup documentation for instructions on backing up your cluster configuration.

Back up the existing management data directory:

cp -r /mnt/mgmt_tgt_mgmt01/data /mnt/mgmt_tgt_mgmt01/data_beegfs_v7_backup_$(date +%Y%m%d)

Run the following commands from a beegfs client and save their output for reference:
```
beegfs-ctl --getentryinfo --verbose /path/to/beegfs/mountpoint
```

If using mirroring, gather detailed state information:

beegfs-ctl --listtargets --longnodes --state --spaceinfo --mirrorgroups --nodetype=meta
beegfs-ctl --listtargets --longnodes --state --spaceinfo --mirrorgroups --nodetype=storage

Prepare your clients for downtime and stop beegfs-client services. For each client, run:
```
systemctl stop beegfs-client
```
For each Pacemaker cluster, disable STONITH. This will allow you to validate the integrity of the cluster after the upgrade without triggering unnecessary node reboots.
```
pcs property set stonith-enabled=false
```
For all Pacemaker clusters in the BeeGFS namespace, use PCS to stop the cluster:
```
pcs cluster stop --all
```

Upgrade the BeeGFS packages

On all file nodes in the cluster, add the BeeGFS v8 package repository for your Linux distribution. Instructions for using the official BeeGFS repositories can be found at the BeeGFS download page. Otherwise, configure your local beegfs mirror repository accordingly.

The following steps walkthrough using the official BeeGFS 8.2 repository on RHEL 9 file nodes. Perform the following steps on all file nodes in the cluster:

Import the BeeGFS GPG key:

rpm --import https://www.beegfs.io/release/beegfs_8.2/gpg/GPG-KEY-beegfs

Import the BeeGFS repository:
```
curl -L -o /etc/yum.repos.d/beegfs-rhel9.repo https://www.beegfs.io/release/beegfs_8.2/dists/beegfs-rhel9.repo
```
Remove any previously configured BeeGFS repositories to avoid conflicts with the new BeeGFS v8 repository.
Clean your package manager cache:
```
dnf clean all
```

On all file nodes, update the BeeGFS packages to BeeGFS 8.2.

dnf update beegfs-mgmtd beegfs-storage beegfs-meta libbeegfs-ib

In a standard cluster, the beegfs-mgmtd package will only update on the first two file nodes.

Upgrade the management database

On one of the file nodes running the BeeGFS management service, perform the following steps to migrate the management database from BeeGFS v7 to v8.

List all NVMe devices and filter for the management target:
```
nvme netapp smdevices | grep mgmt_tgt
```
1. Note the device path from the output.
2. Mount the management target device to the existing management target mount point (replace /dev/nvmeXnY with your device path):
  mount /dev/nvmeXnY /mnt/mgmt_tgt_mgmt01/

Import your BeeGFS 7 management data into the new database format by running:

/opt/beegfs/sbin/beegfs-mgmtd --import-from-v7=/mnt/mgmt_tgt_mgmt01/data/

Expected output:

Created new database version 3 at "/var/lib/beegfs/mgmtd.sqlite". Successfully imported v7 management data from "/mnt/mgmt_tgt_mgmt01/data/".

The automatic import may not succeed in all cases due to stricter validation requirements in BeeGFS v8. For example, if targets are assigned to non-existent storage pools, the import will fail. If the migration fails, do not proceed with the upgrade. Contact NetApp support for assistance with resolving the database migration issues. As an interim solution, you can downgrade the BeeGFS v8 packages and continue running BeeGFS v7 while the issue is addressed.

Move the generated SQLite file to the management service mount:
```
mv /var/lib/beegfs/mgmtd.sqlite /mnt/mgmt_tgt_mgmt01/data/
```
Move the generated beegfs-mgmtd.toml to the management service mount:
```
mv /etc/beegfs/beegfs-mgmtd.toml /mnt/mgmt_tgt_mgmt01/mgmt_config/
```
Preparing the beegfs-mgmtd.toml configuration file will be done after completing the licensing and TLS configuration steps in the next sections.

Configure licensing

Install the beegfs license packages on all nodes that run the beegfs management service. This is typically the first two nodes of the cluster:
```
dnf install libbeegfs-license
```
Download your BeeGFS v8 license file to the management nodes and place it at:
```
/etc/beegfs/license.pem
```

Configure TLS encryption

BeeGFS v8 requires TLS encryption for secure communication between management services and clients. There are three options for configuring TLS encryption on network communications between management services and client services. The recommended and most secure method is to use certificates signed by a trusted Certificate Authority. Alternatively, you can create your own local CA to sign certificates for your BeeGFS cluster. For environments where encryption is not required or for troubleshooting, TLS can be disabled entirely, though this is discouraged as it exposes sensitive information to the network.

Before proceeding, follow the instructions in the Configure TLS Encryption for BeeGFS 8 guide to setup TLS encryption for your environment.

Update management service configuration

Prepare the BeeGFS v8 management service configuration file by manually transferring settings from your BeeGFS v7 configuration file into the /mnt/mgmt_tgt_mgmt01/mgmt_config/beegfs-mgmtd.toml file.

On the management node with the management target mounted, reference the /mnt/mgmt_tgt_mgmt01/mgmt_config/beegfs-mgmtd.conf management service file for BeeGFS 7, then transfer all the settings to the /mnt/mgmt_tgt_mgmt01/mgmt_config/beegfs-mgmtd.toml file. For a basic setup, your beegfs-mgmtd.toml may look like the following:

beemsg-port = 8008
grpc-port = 8010
log-level = "info"
node-offline-timeout = "900s"
quota-enable = false
auth-disable = false
auth-file = "/etc/beegfs/<mgmt_service_ip>_connAuthFile"
db-file = "/mnt/mgmt_tgt_mgmt01/data/mgmtd.sqlite"
license-disable = false
license-cert-file = "/etc/beegfs/license.pem"
tls-disable = false
tls-cert-file = "/etc/beegfs/mgmtd_tls_cert.pem"
tls-key-file = "/etc/beegfs/mgmtd_tls_key.pem"
interfaces = ['i1b:mgmt_1', 'i2b:mgmt_2']

Adjust all paths as necessary to match your environment and TLS configuration.

On each file node running management services, modify your systemd service file to point to the new configuration file location.

sudo sed -i 's|ExecStart=.*|ExecStart=nice -n -3 /opt/beegfs/sbin/beegfs-mgmtd --config-file /mnt/mgmt_tgt_mgmt01/mgmt_config/beegfs-mgmtd.toml|' /etc/systemd/system/beegfs-mgmtd.service

Reload systemd:
```
systemctl daemon-reload
```

For each file node running management services, open port 8010 for the management service's gRPC communication.
1. Add port 8010/tcp to the beegfs zone:
  sudo firewall-cmd --zone=beegfs --permanent --add-port=8010/tcp
2. Reload the firewall to apply the change:
  sudo firewall-cmd --reload

Update the BeeGFS monitor script

The Pacemaker beegfs-monitor OCF script requires updates to support the new TOML configuration format and systemd service management. Update the script on one node in the cluster, then copy the updated script to all other nodes.

Create a backup of the current script:

cp /usr/lib/ocf/resource.d/eseries/beegfs-monitor /usr/lib/ocf/resource.d/eseries/beegfs-monitor.bak.$(date +%F)

Update the management configuration file path from .conf to .toml:

sed -i 's|mgmt_config/beegfs-mgmtd\.conf|mgmt_config/beegfs-mgmtd.toml|' /usr/lib/ocf/resource.d/eseries/beegfs-monitor

Alternatively, manually locate the following block in the script:

case $type in
  management)
    conf_path="${configuration_mount}/mgmt_config/beegfs-mgmtd.conf"
    ;;

And replace it with:

case $type in
  management)
    conf_path="${configuration_mount}/mgmt_config/beegfs-mgmtd.toml"
    ;;

Update the get_interfaces() and get_subnet_ips() functions to support TOML configuration:

Open the script in a text editor:

vi /usr/lib/ocf/resource.d/eseries/beegfs-monitor

Locate the two functions: get_interfaces() and get_subnet_ips().
Delete both entire functions, starting at get_interfaces() to the end of get_subnet_ips().

Copy and paste the following updated functions in their place:

# Return network communication interface name(s) from the BeeGFS resource's connInterfaceFile
get_interfaces() {
  # Determine BeeGFS service network IP interfaces.
  if [ "$type" = "management" ]; then
    interfaces_line=$(grep "^interfaces =" "$conf_path")
    interfaces_list=$(echo "$interfaces_line" | sed "s/.*= \[\(.*\)\]/\1/")
    interfaces=$(echo "$interfaces_list" | tr -d "'" | tr -d " " | tr ',' '\n')

    for entry in $interfaces; do
      echo "$entry" | cut -d ':' -f 1
    done
  else
    connInterfacesFile_path=$(grep "^connInterfacesFile" "$conf_path" | tr -d "[:space:]" | cut -f 2 -d "=")

    if [ -f "$connInterfacesFile_path" ]; then
      while read -r entry; do
        echo "$entry" | cut -f 1 -d ':'
      done < "$connInterfacesFile_path"
    fi
  fi
}

# Return list containing all the BeeGFS resource's usable IP addresses. *Note that these are filtered by the connNetFilterFile entries.
get_subnet_ips() {
  # Determine all possible BeeGFS service network IP addresses.
  if [ "$type" != "management" ]; then
    connNetFilterFile_path=$(grep "^connNetFilterFile" "$conf_path" | tr -d "[:space:]" | cut -f 2 -d "=")

    filter_ips=""
    if [ -n "$connNetFilterFile_path" ] && [ -e $connNetFilterFile_path ]; then
      while read -r filter; do
        filter_ips="$filter_ips $(get_ipv4_subnet_addresses $filter)"
      done < $connNetFilterFile_path
    fi

    echo "$filter_ips"
  fi
}

Save and exit the text editor.
Run the following command to check the script for syntax errors before proceeding. No output indicates the script is syntactically correct.
```
bash -n /usr/lib/ocf/resource.d/eseries/beegfs-monitor
```

Copy the updated beegfs-monitor OCF script to all other nodes in the cluster to ensure consistency:

scp /usr/lib/ocf/resource.d/eseries/beegfs-monitor user@node:/usr/lib/ocf/resource.d/eseries/beegfs-monitor

Bring the cluster back online

Once all the previous upgrade steps have been completed, bring the cluster back online by starting the BeeGFS services on all nodes.
```
pcs cluster start --all
```

Verify the beegfs-mgmtd service started successfully:

journalctl -xeu beegfs-mgmtd

Expected output includes lines such as:

Started Cluster Controlled beegfs-mgmtd.
Loaded config file from "/mnt/mgmt_tgt_mgmt01/mgmt_config/beegfs-mgmtd.toml"
Successfully initialized certificate verification library.
Successfully loaded license certificate: TMP-113489268
Opened database at "/mnt/mgmt_tgt_mgmt01/data/mgmtd.sqlite"
Listening for BeeGFS connections on [::]:8008
Serving gRPC requests on [::]:8010

If errors appear in the journal logs, review the management configuration file paths and ensure all values were correctly transferred from the BeeGFS 7 configuration file.

Run pcs status and verify the cluster is healthy and services are started on their preferred nodes.
Once the cluster is verified to be healthy, re-enable STONITH:
```
pcs property set stonith-enabled=true
```
Proceed to the next section to upgrade the BeeGFS clients on in the cluster and check the BeeGFS cluster's health.

Upgrade BeeGFS clients

After successfully upgrading your cluster to BeeGFS v8, you must also upgrade all BeeGFS clients.

The following steps outline the process to upgrade BeeGFS clients on an Ubuntu-based system.

If not already done, stop the BeeGFS client service:
```
systemctl stop beegfs-client
```
Add the BeeGFS v8 package repository for your Linux distribution. Instructions for using the official BeeGFS repositories can be found at the ^BeeGFS download page. Otherwise, configure your local BeeGFS mirror repository accordingly.

The following steps use the official BeeGFS 8.2 repository on an Ubuntu-based system: