Skip to main content
Enterprise applications

Direct-connect networking

Contributors jfsinmsp
PDFs

System administrators sometimes prefer to simplify their infrastructures by removing network switches from the configuration. This allows creation of a more self-contained solution with fewer components and less space required in the rack. There are some important restrictions that depend on the protocol in use.

Fiber Channel

Starting with ONTAP 9.19.1, direct-connect FC is supported with both traditional FC-SAN (SCSI) and NVMe/FC with select storage system FC adapters. The HBAs will establish a standard point-to-point FC connection.

Prior to ONTAP 9.19.1, ONTAP storage systems require NPIV, which means the storage system must be connected to an FC switch that also supports NPIV.

iSCSI and NVMe/TCP

A host using iSCSI or NVMe/TCP can be directly connected to a storage system and operate normally. The reason is pathing. Direct connections to two different storage controllers results in two independent paths for data flow. The loss of path, port, or controller does not prevent the other path from being used.

NFS

Direct-connected NFS storage can be used, but with a significant limitation - failover will not work without a significant scripting effort, which would be the responsibility of the customer.

The reason nondisruptive failover is complicated with direct-connected NFS storage is the routing that occurs on the local OS. For example, assume a host has an IP address of 192.168.1.1/24 and is directly connected to an ONTAP controller with an IP address of 192.168.1.50/24. During failover, that 192.168.1.50 address can fail over to the other controller, and it will be available to the host, but how does the host detect its presence? The original 192.168.1.1 address still exists on the host NIC that no longer connects to an operational system. Traffic destined for 192.168.1.50 would continue to be sent to an inoperable network port.

The second OS NIC could be configured as 192.168.1.2 and would be capable of communicating with the failed over 192.168.1.50 address, but the local routing tables would have a default of using one and only one address to communicate with the 192.168.1.0/24 subnet. A sysadmin could create a scripting framework that would detect a failed network connection and alter the local routing tables or bring interfaces up and down. The exact procedure would depend on the OS in use.

In practice, NetApp customers do have direct-connected NFS, but normally only for workloads where IO pauses during failovers are acceptable. When hard mounts are used, there should not be any IO errors during such pauses. The IO should hang until services are restored, either by a failback or manual intervention to move IP addresses between NICs on the host.