Configure Oracle Linux 7.x with NVMe-oF for ONTAP storage

12/10/2025 Contributors

Oracle Linux hosts support the NVMe over Fibre Channel (NVMe/FC) and NVMe over TCP (NVMe/TCP) protocols with Asymmetric Namespace Access (ANA). ANA provides multipathing functionality equivalent to asymmetric logical unit access (ALUA) in iSCSI and FCP environments.

Learn how to configure NVMe over Fabrics (NVMe-oF) hosts for Oracle Linux 7.x. For more support and feature information, see Oracle Linux ONTAP support and features.

NVMe-oF with Oracle Linux 7.x has the following known limitations:

SAN booting using the NVMe-oF protocol is not supported.
NetApp sanlun host utility support isn't available for NVMe-oF on an Oracle Linux 7.x host. Instead, you can rely on the NetApp plug-in included in the native nvme-cli package for all NVMe-oF transports.
Native NVMe/FC auto-connect scripts are not available in the nvme-cli package. Use the HBA vendor provided external auto-connect scripts.
Round-robin load balancing is not enabled by default for NVMe multipathing. To enable this functionality, write a udev rule.

Step 1: Install Oracle Linux and NVMe software and verify your configuration

Use the following procedure to validate the minimum supported Oracle Linux 7.x software versions.

Steps

Install Oracle Linux 7.x on the server. After the installation is complete, verify that you are running the specified Oracle Linux 7.x kernel.
```
uname -r
```
Example Oracle Linux kernel version:
```
5.4.17-2011.6.2.el7uek.x86_64
```
Install the nvme-cli package:
```
rpm -qa | grep nvme-cli
```
The following example shows an nvme-cli package version:
```
nvme-cli-1.8.1-3.el7.x86_64
```

Add the following string as a separate udev rule for /lib/udev/rules.d/71-nvme-iopolicy-netapp-ONTAP.rules. This enables round-robin load balancing for NVMe multipath.

cat /lib/udev/rules.d/71-nvme-iopolicy-netapp-ONTAP.rules
Enable round-robin for NetApp ONTAP
ACTION=="add", SUBSYSTEMS=="nvme-subsystem", ATTRS{model}=="NetApp ONTAP Controller", ATTR{iopolicy}="round-robin"

On the Oracle Linux 7.x host, check the hostnqn string at /etc/nvme/hostnqn:
```
cat /etc/nvme/hostnqn
```
The following example shows an hostnqn version:
```
nqn.2014-08.org.nvmexpress:uuid:497ad959-e6d0-4987-8dc2-a89267400874
```

On the ONTAP system, verify that the hostnqn string matches the hostnqn string for the corresponding subsystem on the ONTAP storage system:

*> vserver nvme subsystem host show -vserver vs_nvme_10

Show example

Vserver Subsystem Host NQN
------- --------- -------------------------------------- -----------
ol_157_nvme_ss_10_0
nqn.2014-08.org.nvmexpress:uuid:497ad959-e6d0-4987-8dc2-a89267400874

If the hostnqn strings don't match, use the vserver modify command to update the hostnqn string on your corresponding ONTAP array subsystem to match the hostnqn string from /etc/nvme/hostnqn on the host.

Reboot the host.

Step 2: Configure NVMe/FC

Configure NVMe/FC for a Broadcom/Emulex adapter.

Verify that you're using the supported adapter model:

Display the model names:

cat /sys/class/scsi_host/host*/modelname

You should see the following output:

LPe32002-M2
LPe32002-M2

Display the model descriptions:

cat /sys/class/scsi_host/host*/modeldesc

You should see an output similar to the following example:

Emulex LightPulse LPe32002-M2 2-Port 32Gb Fibre Channel Adapter
Emulex LightPulse LPe32002-M2 2-Port 32Gb Fibre Channel Adapter

Verify that lpfc_enable_fc4_type is set to "3":

cat /sys/module/lpfc/parameters/lpfc_enable_fc4_type

Install the recommended lpfc auto-connect scripts:
```
rpm -ivh nvmefc-connect-12.8.264.0-1.noarch.rpm
```
Verify that the auto-connect scripts are installed:
```
rpm -qa | grep nvmefc
```
You should see the following output:
```
nvmefc-connect-12.8.264.0-1.noarch
```
Verify that your initiator ports are online:
1. Display the port name:
  cat /sys/class/fc_host/host*/port_name
  You should see the following output:
  0x10000090fae0ec61 0x10000090fae0ec62
2. Display the port name:
  cat /sys/class/fc_host/host*/port_state
  You should see the following output:
  Online Online

Verify that the NVMe/FC initiator ports are enabled and that the target ports are visible:

cat /sys/class/scsi_host/host*/nvme_info

Show example

NVME Initiator Enabled
XRI Dist lpfc0 Total 6144 NVME 2947 SCSI 2947 ELS 250
NVME LPORT lpfc0 WWPN x10000090fae0ec61 WWNN x20000090fae0ec61 DID x012000 ONLINE
NVME RPORT WWPN x202d00a098c80f09 WWNN x202c00a098c80f09 DID x010201 TARGET DISCSRVC ONLINE
NVME RPORT WWPN x203100a098c80f09 WWNN x202c00a098c80f09 DID x010601 TARGET DISCSRVC ONLINE

Step 3: Optionally, enable 1MB I/O for NVMe/FC

ONTAP reports a Max Data Transfer Size (MDTS) of 8 in the Identify Controller data. This means the maximum I/O request size can be up to 1MB. To issue I/O requests of size 1MB for a Broadcom NVMe/FC host, you should increase the lpfc value of the lpfc_sg_seg_cnt parameter to 256 from the default value of 64.

These steps don't apply to Qlogic NVMe/FC hosts.

Steps

Set the lpfc_sg_seg_cnt parameter to 256:
```
cat /etc/modprobe.d/lpfc.conf
```
You should see an output similar to the following example:
```
options lpfc lpfc_sg_seg_cnt=256
```
Run the dracut -f command, and reboot the host.

Verify that the value for lpfc_sg_seg_cnt is 256:

cat /sys/module/lpfc/parameters/lpfc_sg_seg_cnt

Step 4: Verify the multipathing configuration

Verify that the in-kernel NVMe multipath status, ANA status, and ONTAP namespaces are correct for the NVMe-oF configuration.

Steps

Verify that the in-kernel NVMe multipath is enabled:
```
cat /sys/module/nvme_core/parameters/multipath
```
You should see the following output:
```
Y
```
Verify that the appropriate NVMe-oF settings (such as, model set to NetApp ONTAP Controller and load balancing iopolicy set to round-robin) for the respective ONTAP namespaces correctly reflect on the host:
1. Display the subsystems:
  cat /sys/class/nvme-subsystem/nvme-subsys*/model
  You should see the following output:
  NetApp ONTAP Controller NetApp ONTAP Controller
2. Display the policy:
  cat /sys/class/nvme-subsystem/nvme-subsys*/iopolicy
  You should see the following output:
  round-robin round-robin

Verify that the namespaces are created and correctly discovered on the host:

nvme list

Show example

Node SN Model Namespace Usage Format FW Rev
---------------- -------------------- -----------------------
/dev/nvme0n1 80BADBKnB/JvAAAAAAAC NetApp ONTAP Controller 1 53.69 GB / 53.69 GB 4 KiB + 0 B FFFFFFFF

Verify that the controller state of each path is live and has the correct ANA status:

nvme list-subsys /dev/nvme0n1

Show example

Nvme-subsysf0 – NQN=nqn.1992-08.com.netapp:sn.341541339b9511e8a9b500a098c80f09:subsystem.ol_157_nvme_ss_10_0
\
+- nvme0 fc traddr=nn-0x202c00a098c80f09:pn-0x202d00a098c80f09 host_traddr=nn-0x20000090fae0ec61:pn-0x10000090fae0ec61 live optimized
+- nvme1 fc traddr=nn-0x207300a098dfdd91:pn-0x207600a098dfdd91 host_traddr=nn-0x200000109b1c1204:pn-0x100000109b1c1204 live inaccessible
+- nvme2 fc traddr=nn-0x207300a098dfdd91:pn-0x207500a098dfdd91 host_traddr=nn-0x200000109b1c1205:pn-0x100000109b1c1205 live optimized
+- nvme3 fc traddr=nn-0x207300a098dfdd91:pn-0x207700a098dfdd91 host traddr=nn-0x200000109b1c1205:pn-0x100000109b1c1205 live inaccessible

Verify that the NetApp plug-in displays the correct values for each ONTAP namespace device:

Column

nvme netapp ontapdevices -o column

Show example

Device   Vserver  Namespace Path             NSID   UUID   Size
-------  -------- -------------------------  ------ ----- -----
/dev/nvme0n1   vs_nvme_10       /vol/rhel_141_vol_10_0/ol_157_ns_10_0    1        55baf453-f629-4a18-9364-b6aee3f50dad   53.69GB

JSON

nvme netapp ontapdevices -o json

Show example

{
   "ONTAPdevices" : [
   {
        Device" : "/dev/nvme0n1",
        "Vserver" : "vs_nvme_10",
        "Namespace_Path" : "/vol/rhel_141_vol_10_0/ol_157_ns_10_0",
         "NSID" : 1,
         "UUID" : "55baf453-f629-4a18-9364-b6aee3f50dad",
         "Size" : "53.69GB",
         "LBA_Data_Size" : 4096,
         "Namespace_Size" : 13107200
    }
]

Step 5: Review the known issues

There are no known issues.

Configure Oracle Linux 7.x with NVMe-oF for ONTAP storage

Creating your file...

Step 1: Install Oracle Linux and NVMe software and verify your configuration

Step 2: Configure NVMe/FC

Step 3: Optionally, enable 1MB I/O for NVMe/FC

Step 4: Verify the multipathing configuration

Step 5: Review the known issues