Skip to main content
BeeGFS on NetApp with E-Series Storage

Technical requirements

Contributors mcwhiteside netapp-jsnyder netapp-jolieg iamjoemccormick

To implement the BeeGFS on NetApp solution, ensure your environment meets the technology requirements outlined in this document.

Requirements overview

This page details the hardware, software, and firmware requirements for implementing BeeGFS on NetApp building blocks. Requirements are organized into the following sections:

Hardware requirements

Before you begin, ensure that your hardware meets the following specifications for a single building block design of the BeeGFS on NetApp solution. The exact components for a particular deployment may vary based on customer requirements. Each file node should meet or exceed these specifications to achieve expected performance.

File node requirements

Quantity Specifications

2

Lenovo ThinkSystem SR665 V3

Hardware:

  • Processors: 2x AMD EPYC 9124 16C 3.0 GHz (configured as two NUMA zones)

  • Memory: 256GB (16x 16GB TruDDR5 4800MHz RDIMM-A)

  • PCIe Expansion: Four PCIe Gen5 x16 slots (two per NUMA zone)

  • HCA Cards: 4x MCX755106AS-HEAT ConnectX-7, NDR200, QSFP112, 2-port, PCIe Gen5 x16, InfiniBand Adapter

  • Storage: Two drives in RAID 1 for OS (1TB 7.2K SATA or better)

  • Management: 1GbE port for in-band OS management

  • BMC: 1GbE BMC with Redfish API for out-of-band server management

  • Power: Dual hot swap power supplies and performance fans

Software and Firmware:

  • Operating System: Red Hat 9.4 Server Physical with High Availability (2 socket)

  • Linux Kernel: 5.14.0-427.42.1.el9_4.x86_64

  • HCA Firmware: FW: 28.46.1006, PXE: 3.8.0100, UEFI: 14.39.0013

Important File nodes require a valid Red Hat Enterprise Linux Server subscription and the Red Hat Enterprise Linux High Availability Add-On.

Block node requirements

Each building block requires two E-Series arrays of the same model. Do not mix array models within the same building block.

Quantity Model Specifications

2

EF600

Hardware:

  • Memory: 256GB (128GB per controller)

  • Adapter: 2-port 200Gb/HDR (NVMe/IB)

  • Drives: Configured to match desired metadata and storage capacity

Firmware:

  • SANtricity OS: 11.90R5

  • NVSRAM: N6000-890834-D03

  • Drive Firmware: Latest available for your drive models (see E-Series disk firmware site)

Storage network switch requirements

Quantity Specifications

1

NVIDIA QM9700 Quantum 2 NDR InfiniBand Switch

The storage network switch must be capable of 200Gb/s InfiniBand speeds. The NVIDIA QM9700 is the recommended switch for the cluster.

Note Alternative switches such as the NVIDIA MQM8700 Quantum may be used but require different cabling.

Cabling requirements

Block node to file node cables

Direct connections from block nodes to file nodes require the following cables:

Quantity Cable Specification Length

8

MCP1650-H001E30 - NVIDIA Passive Copper cable, QSFP56, 200Gb/s

1m

File node to storage switch cables

The following cable types are recommended for the NVIDIA QM9700 Quantum 2 switch. Choose either active fiber or passive copper based on your requirements. Quantities are per building block.

Switch model Cable Type Quantity Part Number

NVIDIA QM9700

Active Fiber (including transceivers)

2

MMA4Z00-NS (multimode, IB/ETH, 800Gb/s 2x400Gb/s Twin-port OSFP)

4

MFP7E20-Nxxx (multimode, 4-channel-to-two 2-channel splitter fiber cable)

8

MMA1Z00-NS400 (multimode, IB/ETH, 400Gb/s Single-port QSFP-112)

Passive Copper

2

MCP7Y40-N002 (NVIDIA passive copper splitter cable, InfiniBand 800Gb/s to 4x 200Gb/s, OSFP to 4x QSFP112)

Software deployment requirements

The following table lists the software requirements deployed automatically as part of the Ansible-based BeeGFS deployment.

Software Version

BeeGFS

8.2.2

Corosync

3.1.8-1

Pacemaker

2.1.7-5.2

Fence Agents (redfish/apc)

4.10.0-62

InfiniBand / RDMA Drivers

DOCA-OFED-25.04-0.6.1

Ansible control node requirements

The BeeGFS on NetApp solution is deployed and managed from an Ansible control node. For more information, see the Ansible documentation.

The software requirements listed in the following tables are specific to the version of the NetApp BeeGFS Ansible collection listed below.

Software Version

Ansible-core

>= 2.19.0

Python

3.12

Additional Python packages

Cryptography-46.0.3, netaddr-1.3.0, ipaddr-2.2.0

NetApp E-Series BeeGFS Ansible Collection

4.0.0