Hardware details
Suggest changes
PDFs
The following section covers details about the NetApp AFX cluster hardware. For the most recent information about NetApp AFX hardware, see https://hwu.netapp.com.
Supported hardware
For the latest information on supported hardware for NetApp AFX, check https://hwu.netapp.com.
Nodes
NetApp AFX nodes include the AFX 1K (based on the AFF A1K) and the AFX 2K. These nodes have no on-board disk for storage and are intended to be modular to easily add and remove nodes based on performance requirements. Each node uses 2U of rack space and increases performance linearly as they are added to an AFX cluster.
NetApp AFX 1K node details
NetApp AFX 2K node details
Hardware slots
NetApp AFX 1K nodes use the following slot assignments.
-
Slot 1 is dedicated to HA replication
-
Slot 7 is dedicated to cluster replication
-
Slots 10 and 11 are dedicated to storage shelf communication
NetApp AFX 2K nodes use the following slot assignments.
-
Slot 1 is dedicated to HA replication
-
Slot 2 is dedicated to cluster replication
-
Slots 8B and 10A are dedicated to storage shelf communication
Shelves
NetApp AFX shelves use the same enclosures that the AFF systems leverage. The main difference in shelves in AFX is in the module used. The NSM140 modules provide enhanced performance capabilities and help make disaggregated ONTAP possible. A few key considerations:
-
Only fully populated shelves supported.
-
Shelves are automatically detected by NetApp AFX when plugged in.
-
No shelf removal support currently.
NetApp AFX shelf enclosure details
Switches
NetApp AFX still makes use of backend cluster switches for intracluster communication, such as cluster database replication, remote data operations, storage operations, and NVRAM mirroring. Functionally, the only differences between unified ONTAP and NetApp AFX cluster switches are:
-
400GbE support
-
New HA VLAN
-
For more information, see the Cisco switches data sheet.
NetApp AFX cluster switches
Cluster switch connectivity
NetApp AFX heavily leverages backend cluster switches for many of its major architectural concepts. For instance, cluster interfaces, storage adapters, storage shelves, and NVRAM cards all plug into the cluster switches. The cluster switches are 400GB capable. Backend cluster network ports support only 100GB connections to the cluster switches on the AFX 1K platform and support 400GB connections on the AFX 2K. As a result, AFX 1K backend node connections use 4 x 100GB breakout cables to reduce the number of ports used on the switches. The AFX 2K uses a 1:1 400GB connection to the switches, with 6 ports per node (2 HA, 2 cluster, 2 storage). Storage shelf modules still use 100GB interfaces; for instance, 16 x 100GB storage shelf module ports use just 4 ports on the switches, while the 8 total ports on AFX 1K nodes use 2 switch ports.
NetApp AFX switch cabling
Switch selection guidance
Add up the 400GbE ports required on each switch, then select the smallest switch class that covers that total. NetApp AFX uses a pair of cluster switches; size each switch independently. Two ports on each switch are reserved for ISL (9332: ports 31–32; 9364: ports 63–64), which leaves 30 usable ports on the 9332 and 62 usable ports on the 9364.
Port counts per switch:
-
AFX 1K: 1 × 400GbE port per node (four 100GB node ports on a 4 x 100GB breakout)
-
AFX 2K: 3 × 400GbE ports per node per switch (6 ports per node at 1:1 400GB: 2 HA, 2 cluster, 2 storage)
-
Shelves: 2 × 400GbE ports per shelf (eight 100GB shelf ports per switch on 4 x 100GB breakouts)
| Combined 400GbE ports required (per switch) | Switch |
|---|---|
Fits in 32 ports (30 usable after ISL) |
Cisco Nexus 9332D-GX2B (1U, 32 x 400GbE) |
Fits in 64 ports (62 usable after ISL) |
Cisco Nexus 9364D-GX2A (2U, 64 x 400GbE) |
More than 64 ports |
Cisco Nexus 9808, or as specified in Hardware Universe |
32-node clusters
A 32-node cluster (the current node maximum; see Maximums and limits) consumes 32 × 400GbE ports per switch on AFX 1K and 96 × 400GbE ports per switch on AFX 2K (32 × 3), before shelves.
| Switch | AFX 1K (32 nodes) | AFX 2K (32 nodes) |
|---|---|---|
Cisco Nexus 9332D-GX2B |
Not supported (32 node ports exceed 30 usable ports) |
Not supported |
Cisco Nexus 9364D-GX2A |
32 nodes and 8–12 shelves (4:1 to 8:3) |
Not supported (96 node ports exceed 62 usable ports) |
Cisco Nexus 9808 |
32 nodes and 8–12 shelves (4:1 to 8:3) |
32 nodes and 8–12 shelves (4:1 to 8:3) |
Maximum node count per switch
Size to a node-to-shelf ratio of at least 4:1 and at most 8:3 (see Node to shelf ratios), even node counts (HA pairs), and a minimum of 2 shelves. The following combinations are the maximum node counts that fit each switch in that range.
| Switch | AFX 1K (max nodes + shelves) | AFX 2K (max nodes + shelves) |
|---|---|---|
Cisco Nexus 9332D-GX2B |
20 nodes + 5 shelves (4:1; 8:3 does not fit remaining ports) |
8 nodes + 2–3 shelves |
Cisco Nexus 9364D-GX2A |
32 nodes + 8–12 shelves |
16 nodes + 4–6 shelves |
Cisco Nexus 9808 |
32 nodes + 8–12 shelves |
32 nodes + 8–12 shelves |
For the latest supported switch models and limits, see Hardware Universe and the AFX cabling documentation.
Disk types and sizes
NetApp AFX currently only supports NVMe attached SSDs in the following sizes:
-
15.3TB
-
30.1TB
-
60.6TB
TLC vs. QLC
NetApp AFX can leverage both TLC and QLC flash types. The 15.3TB drives are TLC models, while drives larger than 30.1TB will be QLC. Regardless of media type, the NVIDIA SuperPod certification performance standards can be met.
All drives used with NetApp AFX are performance-rated drives, and both QLC and TLC will perform nearly identically for read traffic. QLC lags a bit behind write performance as compared to TLC and can experience slightly more wear-leveling with write-heavy workloads.
For performance numbers, see Per-shelf performance.
When choosing the drive type in use, consider what workloads the storage will be hosting and the performance/capacity density tradeoffs involved.