Access NVMe volumes for physical NVMe device targets
Suggest changes
PDFs
You can configure the I/O directed to the device target based on which OS (and by extension multipathing method) you are using.
For RHEL 8, RHEL 9, and SLES 15, I/O is directed to the physical NVMe device targets by the Linux host. A native NVMe multipathing solution manages the physical paths underlying the single apparent physical device displayed by the host.
Physical NVMe devices are I/O targets
It is best practice to run I/O to the links in /dev/disk/by-id/nvme-eui.[uuid#] rather than directly to the physical nvme device path /dev/nvme[subsys#]n[id#]. The link between these two locations can be found using the following command:
# ls /dev/disk/by-id/ -l lrwxrwxrwx 1 root root 13 Oct 18 15:14 nvme-eui.0000320f5cad32cf00a0980000af4112 -> ../../nvme0n1
I/O run to /dev/disk/by-id/nvme-eui.[uuid#] will be passed directly through /dev/nvme[subsys#]n[id#] which has all paths virtualized underneath it using the Native NVMe multipathing solution.
You can view your paths by running:
# nvme list-subsys
Example output:
nvme-subsys0 - NQN=nqn.1992-08.com.netapp:5700.600a098000a522500000000589aa8a6 \ +- nvme0 rdma traddr=192.4.21.131 trsvcid=4420 live +- nvme1 rdma traddr=192.4.22.141 trsvcid=4420 live
If you specify a physical nvme device when using the 'nvme list-subsys' command, it provides additional information about the paths to that namespace:
# nvme list-subsys /dev/nvme0n1 nvme-subsys0 - NQN=nqn.1992-08.com.netapp:5700.600a098000af44620000000058d5dd96 \ +- nvme0 rdma traddr=192.168.130.101 trsvcid=4420 live non-optimized +- nvme1 rdma traddr=192.168.131.101 trsvcid=4420 live non-optimized +- nvme2 rdma traddr=192.168.130.102 trsvcid=4420 live optimized +- nvme3 rdma traddr=192.168.131.102 trsvcid=4420 live optimized
There are also hooks into the multipath commands to allow you to view your path information for native failover through them as well:
#multipath -ll
|
To view the path information, the following must be set in /etc/multipath.conf: |
defaults {
enable_foreign nvme
}
Example output:
eui.0000a0335c05d57a00a0980000a5229d [nvme]:nvme0n9 NVMe,Netapp E-Series,08520001 size=4194304 features='n/a' hwhandler='ANA' wp=rw |-+- policy='n/a' prio=50 status=optimized | `- 0:0:1 nvme0c0n1 0:0 n/a optimized live `-+- policy='n/a' prio-10 status=non-optimized `- 0:1:1 nvme0c1n1 0:0 n/a non-optimized live