Skip to main content
BeeGFS on NetApp with E-Series Storage

Technical requirements

Contributors mcwhiteside netapp-jsnyder netapp-jolieg iamjoemccormick
PDFs

To implement the BeeGFS on NetApp solution, ensure your environment meets the technology requirements outlined in this document.

Hardware requirements

Before you begin, ensure that your hardware meets the following specifications for a single second-generation building block design of the BeeGFS on NetApp solution. The exact components for a particular deployment may vary based on customer requirements.

Quantity Hardware component Requirements

2

BeeGFS file nodes

Each file node should meet or exceed the specifications of the recommended file nodes to achieve expected performance.

Recommended file node options:

  • Lenovo ThinkSystem SR665 V3

    • Processors: 2x AMD EPYC 9124 16C 3.0 GHz (configured as two NUMA zones).

    • Memory: 256GB (16x 16GB TruDDR5 4800MHz RDIMM-A)

    • PCIe Expansion: Four PCIe Gen5 x16 slots (two per NUMA zone)

    • Miscellaneous:

      • Two drives in RAID 1 for OS (1TB 7.2K SATA or better)

      • 1GbE port for in-band OS management

      • 1GbE BMC with Redfish API for out-of-band server management

      • Dual hot swap power supplies and performance fans

  • Lenovo ThinkSystem SR665

    • Processors: 2x AMD EPYC 7343 16C 3.2 GHz (configured as two NUMA zones).

    • Memory: 256GB (16x 16GB TruDDR4 3200MHz RDIMM-A)

    • PCIe Expansion: Four PCIe Gen4 x16 slots (two per NUMA zone)

    • Miscellaneous:

      • Two drives in RAID 1 for OS (1TB 7.2K SATA or better)

      • 1GbE port for in-band OS management

      • 1GbE BMC with Redfish API for out-of-band server management

      • Dual hot swap power supplies and performance fans

2

E-Series block nodes (EF600 array)

Memory: 256GB (128GB per controller).
Adapter: 2-port 200Gb/HDR (NVMe/IB).
Drives: Configured to match desired metadata and storage capacity.

8

InfiniBand host card adapters (for file nodes).

Host card adapters will vary based on the server model used for the file node. Recommendations for verified file nodes include:

  • Lenovo ThinkSystem SR665 V3 Server:

    • MCX755106AS-HEAT ConnectX-7, NDR200, QSFP112, 2-port, PCIe Gen5 x16, InfiniBand Adapter

  • Lenovo ThinkSystem SR665 Server:

    • MCX653106A-HDAT ConnectX-6, HDR, QSFP-56, 2-port, PCIe Gen4 x16, InfiniBand Adapter

1

Storage network switch

The storage network switch must be capable of 200Gb/s InfiniBand speeds. Recommended switch models include:

  • NVIDIA QM9700 Quantum 2 NDR InfiniBand switch

  • NVIDIA MQM8700 Quantum HDR InfiniBand switch

Cabling requirements

Direct connections from block nodes to file nodes.

Quantity Part Number Length

8

MCP1650-H001E30 (NVIDIA Passive Copper cable, QSFP56, 200Gb/s)

1m

Connections from file nodes to the storage network switch.
Select the appropriate cable option from the following table according to your InfiniBand storage switch.
The recommended cable length is 2m; however, this may vary based on the customer's environment.

Switch model Quantity Cable Type Part Number

NVIDIA QM9700

4

Active Fiber

MFA7U10-H002 (NVIDIA Active Fiber cable, InfiniBand 400Gb/s to 2x 200Gb/s, OSFP to 2x QSFP56)

NVIDIA QM9700

4

Passive Copper

MCP7Y60-H002 (NVIDIA Passive Copper cable, InfiniBand 400Gb/s to 2x 200Gb/s, OSFP to 2x QSFP56)

NVIDIA MQM8700

8

Active Fiber

MFS1S00-H003E (NVIDIA Active Fiber cable, InfiniBand 200Gb/s, QSFP56)

NVIDIA MQM8700

8

Passive Copper

MCP1650-H002E26 (NVIDIA Passive Copper cable, InfiniBand 200Gb/s, QSFP56)

Software requirements

For predictable performance and reliability, releases of the BeeGFS on NetApp solution are tested with specific versions of the software components required to implement the solution.

File node requirements

Software Version

RedHat Enterprise Linux

RedHat 9.3 Server Physical with High Availability (2 socket).

Important File nodes require a valid RedHat Enterprise Linux Server subscription and the Red Hat Enterprise Linux High Availability Add-On.

Linux Kernel

5.14.0-362.24.1.el9_3.x86_64

InfiniBand / RDMA Drivers

MLNX_OFED_LINUX-23.10-3.2.2.0-LTS

HCA Firmware

ConnectX-7 HCA Firmware
FW: 28.39.1002
PXE: 3.7.0201
UEFI: 14.32.0012

ConnectX-6 HCA Firmware
FW: 20.31.1014
PXE: 3.6.0403
UEFI: 14.24.0013

EF600 block node requirements

Software Version

SANtricity OS

11.80.0

NVSRAM

N6000-880834-D08.dlp

Drive Firmware

Latest available for the drive models in use.

Software deployment requirements

The following table lists the software requirements deployed automatically as part of the Ansible-based BeeGFS deployment.

Software Version

BeeGFS

7.4.4

Corosync

3.1.5-4

Pacemaker

2.1.4-5

OpenSM

opensm-5.17.2 (from MLNX_OFED_LINUX-23.10-3.2.2.0-LTS)

Ansible control node requirements

The BeeGFS on NetApp solution is deployed and managed from an Ansible control node. For more information, see the Ansible documentation.

The software requirements listed in the following tables are specific to the version of the NetApp BeeGFS Ansible collection listed below.

Software Version

Ansible

6.x
When installed through pip: ansible-6.0.0 and ansible-core >= 2.13.0

Python

3.9 (or later)

Additional Python packages

Cryptography-43.0.0, netaddr-1.3.0, ipaddr-2.2.0

NetApp E-Series BeeGFS Ansible Collection

3.2.0