English

NVMe/FC Host Configuration for Windows Server 2012 R2 with ONTAP

Contributors netapp-aherbin netapp-cplumer Download PDF of this page

Supportability

NVMe/FC is supported on ONTAP 9.7 or later for Windows Server 2016.

Note that the Broadcom initiator can serve both NVMe/FC and FCP traffic through the same 32G FC adapter ports. For FCP and FC/NVMe, use MSDSM as the Microsoft Multipath I/O (MPIO) option.

See the Hardware Universe for a list of supported FC adapters and controllers. For the most current list of supported configurations & versions, see the NetApp Interoperability Matrix.

Known limitations

Windows Failover Cluster (WFC) is not supported with ONTAP NVMe/FC because ONTAP does not currently support persistent reservations with NVMe/FC.

The external driver shipped by Broadcom for Windows NVMe/FC is not a true NVMe/FC driver but a translational SCSI ⇄ NVMe driver. This translational overhead does not necessarily impact performance, but it does negate the performance benefits of NVMe/FC. Thus, on Windows servers, NVMe/FC and FCP performance is the same, unlike on other operating systems such as Linux, where NVMe/FC performance is significantly better than that of FCP.

Enabling NVMe/FC on a Windows initiator host

Follow these steps to enable FC/NVMe on the Windows initiator host:

  1. Install OneCommand Manager utility on the Windows host.

  2. On each of the HBA initiator ports, set the following HBA driver parameters:

    • EnableNVMe = 1

    • NVMEMode = 0

    • LimTransferSize=1

  3. Reboot the host.

Configuring the Broadcom FC adapter in Windows for NVMe/FC

With the Broadcom adapter for FC/NVMe in a Windows environment, a hostnqn is associated with each host bus adapter (HBA) port. The hostnqn is formatted as follows.

Example
nqn.2017-01.com.broadcom:ecd:nvmf:fc:100000109b1b9765
nqn.2017-01.com.broadcom:ecd:nvmf:fc:100000109b1b9766

Enabling MPIO for NVMe devices on the Windows host

  1. Install Windows Host Utility Kit 7.1 to set the driver parameters that are common to both FC and NVMe.

  2. Open the MPIO properties.

  3. From the Discover Multi-Paths tab, add the device ID listed for NVMe.

    MPIO becomes aware of the NVMe devices, which are visible under disk management.

  4. Open Disk Management and go to Disk Properties.

  5. From the MPIO tab, click Details.

  6. Set the following MSDSM settings:

    • PathVerifiedPeriod: 10

    • PathVerifyEnabled: Enable

    • RetryCount: 6

    • RetryInterval: 1

    • PDORemovedPeriod: 130

  7. Select the MPIO Policy Round Robin with Subset.

  8. Change the registry values:

    HKLM\SYSTEM\CurrentControlSet\Services\mpio\Parameters\PathRecoveryInterval DWORD -> 30
    
    HKLM\SYSTEM\CurrentControlSet\Services\mpio \Parameters\ UseCustomPathRecoveryInterval  DWORD-> 1
  9. Reboot the host.

    The NVMe configuration is now complete on the Windows host.

Validating NVMe/FC

  1. Validate that the Port Type is FC+NVMe.

    Now that NVMe is enabled, you should see the Port Type listed as FC+NVMe, as follows.

    PS C:\Program Files\Emulex\Util\OCManager> .\hbacmd listhba
    
    Manageable HBA List
    
    Port WWN       : 10:00:00:10:9b:1b:97:65
    Node WWN       : 20:00:00:10:9b:1b:97:65
    Fabric Name    : 10:00:c4:f5:7c:a5:32:e0
    Flags          : 8000e300
    Host Name      : INTEROP-57-159
    Mfg            : Emulex Corporation
    Serial No.     : FC71367217
    Port Number    : 0
    Mode           : Initiator
    PCI Bus Number : 94
    PCI Function   : 0
    Port Type      : FC+NVMe
    Model          : LPe32002-M2
    
    Port WWN       : 10:00:00:10:9b:1b:97:66
    Node WWN       : 20:00:00:10:9b:1b:97:66
    Fabric Name    : 10:00:c4:f5:7c:a5:32:e0
    Flags          : 8000e300
    Host Name      : INTEROP-57-159
    Mfg            : Emulex Corporation
    Serial No.     : FC71367217
    Port Number    : 1
    Mode           : Initiator
    PCI Bus Number : 94
    PCI Function   : 1
    Port Type      : FC+NVMe
    Model          : LPe32002-M2
  2. Validate that NVMe/FC subsystems have been discovered.

    The nvme-list command lists the NVMe/FC discovered subsystems.

    PS C:\Program Files\Emulex\Util\OCManager> .\hbacmd nvme-list 10:00:00:10:9b:1b:97:65
    
    Discovered NVMe Subsystems for 10:00:00:10:9b:1b:97:65
    
    NVMe Qualified Name     :  nqn.1992-08.com.netapp:sn.a3b74c32db2911eab229d039ea141105:subsystem.win_nvme_interop-57-159
    Port WWN                :  20:09:d0:39:ea:14:11:04
    Node WWN                :  20:05:d0:39:ea:14:11:04
    Controller ID           :  0x0180
    Model Number            :  NetApp ONTAP Controller
    Serial Number           :  81CGZBPU5T/uAAAAAAAB
    Firmware Version        :  FFFFFFFF
    Total Capacity          :  Not Available
    Unallocated Capacity    :  Not Available
    
    NVMe Qualified Name     :  nqn.1992-08.com.netapp:sn.a3b74c32db2911eab229d039ea141105:subsystem.win_nvme_interop-57-159
    Port WWN                :  20:06:d0:39:ea:14:11:04
    Node WWN                :  20:05:d0:39:ea:14:11:04
    Controller ID           :  0x0181
    Model Number            :  NetApp ONTAP Controller
    Serial Number           :  81CGZBPU5T/uAAAAAAAB
    Firmware Version        :  FFFFFFFF
    Total Capacity          :  Not Available
    Unallocated Capacity    :  Not Available
    Note: At present Namespace Management is not supported by NetApp Arrays.
    PS C:\Program Files\Emulex\Util\OCManager> .\hbacmd nvme-list 10:00:00:10:9b:1b:97:66
    
    Discovered NVMe Subsystems for 10:00:00:10:9b:1b:97:66
    
    NVMe Qualified Name     :  nqn.1992-08.com.netapp:sn.a3b74c32db2911eab229d039ea141105:subsystem.win_nvme_interop-57-159
    Port WWN                :  20:07:d0:39:ea:14:11:04
    Node WWN                :  20:05:d0:39:ea:14:11:04
    Controller ID           :  0x0140
    Model Number            :  NetApp ONTAP Controller
    Serial Number           :  81CGZBPU5T/uAAAAAAAB
    Firmware Version        :  FFFFFFFF
    Total Capacity          :  Not Available
    Unallocated Capacity    :  Not Available
    
    NVMe Qualified Name     :  nqn.1992-08.com.netapp:sn.a3b74c32db2911eab229d039ea141105:subsystem.win_nvme_interop-57-159
    Port WWN                :  20:08:d0:39:ea:14:11:04
    Node WWN                :  20:05:d0:39:ea:14:11:04
    Controller ID           :  0x0141
    Model Number            :  NetApp ONTAP Controller
    Serial Number           :  81CGZBPU5T/uAAAAAAAB
    Firmware Version        :  FFFFFFFF
    Total Capacity          :  Not Available
    Unallocated Capacity    :  Not Available
    
    Note: At present Namespace Management is not supported by NetApp Arrays.
  3. Validate that namespaces have been created.

    The nvme-list-ns command lists the namespaces for a specified NVMe target that lists the namespaces connected to the host.

    PS C:\Program Files\Emulex\Util\OCManager> .\HbaCmd.exe nvme-list-ns 10:00:00:10:9b:1b:97:66 20:08:d0:39:ea:14:11:04 nq
    .1992-08.com.netapp:sn.a3b74c32db2911eab229d039ea141105:subsystem.win_nvme_interop-57-159 0
    
    
    Active Namespaces (attached to controller 0x0141):
    
                                           SCSI           SCSI           SCSI
       NSID           DeviceName        Bus Number    Target Number     OS LUN
    -----------  --------------------  ------------  ---------------   ---------
    0x00000001   \\.\PHYSICALDRIVE9         0               1              0
    0x00000002   \\.\PHYSICALDRIVE10        0               1              1
    0x00000003   \\.\PHYSICALDRIVE11        0               1              2
    0x00000004   \\.\PHYSICALDRIVE12        0               1              3
    0x00000005   \\.\PHYSICALDRIVE13        0               1              4
    0x00000006   \\.\PHYSICALDRIVE14        0               1              5
    0x00000007   \\.\PHYSICALDRIVE15        0               1              6
    0x00000008   \\.\PHYSICALDRIVE16        0               1              7