Skip to main content
NetApp artificial intelligence solutions

Rack and Cable Lustre Hardware

PDFs

Rack the Lustre servers and NetApp EF80 arrays, then cable the backend NVMe-oF and frontend LNet fabrics for each building block.

Before you begin

Review the HCA counts, PCIe slot requirements, path requirements, and MTU guidance in Hardware components.

Steps

  1. Rack and cable all Lustre servers and E-Series storage arrays according to the building block topology.

    Labeled Lustre building block stacked with two OSS/MDS servers and two EF80 arrays

  2. For each EF80 array, cable controller A port 4a to controller B port 4a, and controller A port 4b to controller B port 4b. These dedicated connections provide inter-controller mirroring and are not used for host traffic. See Cable the EF50 and EF80 inter-controller mirroring connections.

  3. Cable the backend NVMe-oF network between servers and arrays according to the EF80 six-HCA topology.

    EF80 six-HCA backend cabling (RoCE or InfiniBand)

    EF80 six-HCA backend NVMe-oF cabling between two OSS/MDS server nodes and two EF80 arrays

  4. Connect the four frontend LNet interfaces (the i1 and i4 HCAs) on each Lustre server and the Lustre clients to the dedicated InfiniBand or RoCE switch fabric.

  5. When using RoCE, configure the fabric for lossless operation with priority flow control (PFC). The Ansible playbooks configure lossless RoCE on all interfaces in the building block.

After you finish

Prepare the servers, arrays, and Ansible control node.