Skip to main content

Set queue depths on SAN hosts

Contributors netapp-aherbin netapp-ahibbard
PDFs

You might need to change the queue depths on your host to achieve the maximum values for ITNs per node and FC port fan-in.

AIX hosts

You can change the queue depth on AIX hosts using the chdev command. Changes made using the chdev command persist across reboots.

Examples:

  • To change the queue depth for the hdisk7 device, use the following command:

    chdev -l hdisk7 -a queue_depth=32

  • To change the queue depth for the fcs0 HBA, use the following command:

    chdev -l fcs0 -a num_cmd_elems=128

    The default value for num_cmd_elems is 200. The maximum value is 2,048.

    Note

    It might be necessary to take the HBA offline to change num_cmd_elems and then bring it back online using the rmdev -l fcs0 -R and makdev -l fcs0 -P commands.

HP-UX hosts

You can change the LUN or device queue depth on HP-UX hosts using the kernel parameter scsi_max_qdepth. You can change the HBA queue depth using the kernel parameter max_fcp_reqs.

  • The default value for scsi_max_qdepth is 8. The maximum value is 255.

    scsi_max_qdepth can be dynamically changed on a running system using the -u option on the kmtune command. The change will be effective for all devices on the system. For example, use the following command to increase the LUN queue depth to 64:

    kmtune -u -s scsi_max_qdepth=64

    It is possible to change queue depth for individual device files using the scsictl command. Changes using the scsictl command are not persistent across system reboots. To view and change the queue depth for a particular device file, execute the following command:

    scsictl -a /dev/rdsk/c2t2d0

    scsictl -m queue_depth=16 /dev/rdsk/c2t2d0

  • The default value for max_fcp_reqs is 512. The maximum value is 1024.

    The kernel must be rebuilt and the system must be rebooted for changes to max_fcp_reqs to take effect. To change the HBA queue depth to 256, for example, use the following command:

    kmtune -u -s max_fcp_reqs=256

Solaris hosts

You can set the LUN and HBA queue depth for your Solaris hosts.

  • For LUN queue depth: The number of LUNs in use on a host multiplied by the per-LUN throttle (lun-queue-depth) must be less than or equal to the tgt-queue-depth value on the host.

  • For queue depth in a Sun stack: The native drivers do not allow for per LUN or per target max_throttle settings at the HBA level. The recommended method for setting the max_throttle value for native drivers is on a per-device type (VID_PID) level in the /kernel/drv/sd.conf and /kernel/drv/ssd.conf files. The host utility sets this value to 64 for MPxIO configurations and 8 for Veritas DMP configurations.

Steps

  1. # cd/kernel/drv

  2. # vi lpfc.conf

  3. Search for /tft-queue (/tgt-queue)

    tgt-queue-depth=32

    Note

    The default value is set to 32 at installation.

  4. Set the desired value based on the configuration of your environment.

  5. Save the file.

  6. Reboot the host using the sync; sync; sync; reboot -- -r command.

VMware hosts for a QLogic HBA

Use the esxcfg-module command to change the HBA timeout settings. Manually updating the esx.conf file is not recommended.

Steps

  1. Log on to the service console as the root user.

  2. Use the #vmkload_mod -l command to verify which Qlogic HBA module is currently loaded.

  3. For a single instance of a Qlogic HBA, run the following command:

    #esxcfg-module -s ql2xmaxqdepth=64 qla2300_707

    Note

    This example uses qla2300_707 module. Use the appropriate module based on the output of vmkload_mod -l.

  4. Save your changes using the following command:

    #/usr/sbin/esxcfg-boot -b

  5. Reboot the server using the following command:

    #reboot

  6. Confirm the changes using the following commands:

    1. #esxcfg-module -g qla2300_707

    2. qla2300_707 enabled = 1 options = 'ql2xmaxqdepth=64'

VMware hosts for an Emulex HBA

Use the esxcfg-module command to change the HBA timeout settings. Manually updating the esx.conf file is not recommended.

Steps

  1. Log on to the service console as the root user.

  2. Use the #vmkload_mod -l grep lpfc command to verify which Emulex HBA is currently loaded.

  3. For a single instance of an Emulex HBA, enter the following command:

    #esxcfg-module -s lpfc0_lun_queue_depth=16 lpfcdd_7xx

    Note

    Depending on the model of the HBA, the module can be either lpfcdd_7xx or lpfcdd_732. The above command uses the lpfcdd_7xx module. You should use the appropriate module based on the outcome of vmkload_mod -l.

    Running this command will set the LUN queue depth to 16 for the HBA represented by lpfc0.

  4. For multiple instances of an Emulex HBA, run the following command:

    a esxcfg-module -s "lpfc0_lun_queue_depth=16 lpfc1_lun_queue_depth=16" lpfcdd_7xx

    The LUN queue depth for lpfc0 and the LUN queue depth for lpfc1 is set to 16.

  5. Enter the following command:

    #esxcfg-boot -b

  6. Reboot using #reboot.

Windows hosts for an Emulex HBA

On Windows hosts, you can use the LPUTILNT utility to update the queue depth for Emulex HBAs.

Steps

  1. Run the LPUTILNT utility located in the C:\WINNT\system32 directory.

  2. Select Drive Parameters from the menu on the right side.

  3. Scroll down and double-click QueueDepth.

    Note

    If you are setting QueueDepth greater than 150, the following Windows Registry value also need to be increased appropriately:

    HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\lpxnds\Parameters\Device\NumberOfRequests

Windows hosts for a Qlogic HBA

On Windows hosts, you can use theand the SANsurfer HBA manager utility to update the queue depths for Qlogic HBAs.

Steps

  1. Run the SANsurfer HBA manager utility.

  2. Click on HBA port > Settings.

  3. Click Advanced HBA port settings in the list box.

  4. Update the Execution Throttle parameter.

Linux hosts for Emulex HBA

You can update the queue depths of an Emulex HBA on a Linux host. To make the updates persistent across reboots, you must then create a new RAM disk image and reboot the host.

Steps

  1. Identify the queue depth parameters to be modified:

    modinfo lpfc|grep queue_depth

    The list of queue depth parameters with their description is displayed. Depending on your operating system version, you can modify one or more of the following queue depth parameters:

    • lpfc_lun_queue_depth: Maximum number of FC commands that can be queued to a specific LUN (uint)

    • lpfc_hba_queue_depth: Maximum number of FC commands that can be queued to an lpfc HBA (uint)

    • lpfc_tgt_queue_depth: Maximum number of FC commands that can be queued to a specific target port (uint)

      The lpfc_tgt_queue_depth parameter is applicable only for Red Hat Enterprise Linux 7.x systems, SUSE Linux Enterprise Server 11 SP4 systems and 12.x systems.

  2. Update the queue depths by adding the queue depth parameters to the /etc/modprobe.conf file for a Red Hat Enterprise Linux 5.x system and to the /etc/modprobe.d/scsi.conf file for a Red Hat Enterprise Linux 6.x or 7.x system, or a SUSE Linux Enterprise Server 11.x or 12.x system.

    Depending on your operating system version, you can add one or more of the following commands:

    • options lpfc lpfc_hba_queue_depth=new_queue_depth

    • options lpfc lpfc_lun_queue_depth=new_queue_depth

    • options lpfc_tgt_queue_depth=new_queue_depth

  3. Create a new RAM disk image, and then reboot the host to make the updates persistent across reboots.

    For more information, see the System administration for your version of Linux operating system.

  4. Verify that the queue depth values are updated for each of the queue depth parameter that you have modified:

    cat /sys/class/scsi_host/host_number/lpfc_lun_queue_depthcat /sys/class/scsi_host/host_number/lpfc_tgt_queue_depthcat /sys/class/scsi_host/host_number/lpfc_hba_queue_depth

    root@localhost ~]#cat /sys/class/scsi_host/host5/lpfc_lun_queue_depth
      30

    The current value of the queue depth is displayed.

Linux hosts for QLogic HBA

You can update the device queue depth of a QLogic driver on a Linux host. To make the updates persistent across reboots, you must then create a new RAM disk image and reboot the host. You can use the QLogic HBA management GUI or command-line interface (CLI) to modify the QLogic HBA queue depth.

This task shows how to use the QLogic HBA CLI to modify the QLogic HBA queue depth

Steps

  1. Identify the device queue depth parameter to be modified:

    modinfo qla2xxx | grep ql2xmaxqdepth

    You can modify only the ql2xmaxqdepth queue depth parameter, which denotes the maximum queue depth that can be set for each LUN. The default value is 64 for RHEL 7.5 and later. The default value is 32 for RHEL 7.4 and earlier.

    root@localhost ~]# modinfo qla2xxx|grep ql2xmaxqdepth
parm:       ql2xmaxqdepth:Maximum queue depth to set for each LUN. Default is 64. (int)

  2. Update the device queue depth value:

    • If you want to make the modifications persistent, perform the following steps:

      1. Update the queue depths by adding the queue depth parameter to the /etc/modprobe.conf file for a Red Hat Enterprise Linux 5.x system and to the /etc/modprobe.d/scsi.conf file for a Red Hat Enterprise Linux 6.x or 7.x system, or a SUSE Linux Enterprise Server 11.x or 12.x system: options qla2xxx ql2xmaxqdepth=new_queue_depth

      2. Create a new RAM disk image, and then reboot the host to make the updates persistent across reboots.

        For more information, see the System administration for your version of Linux operating system.

    • If you want to modify the parameter only for the current session, run the following command:

      echo new_queue_depth > /sys/module/qla2xxx/parameters/ql2xmaxqdepth

      In the following example, the queue depth is set to 128.

      echo 128 > /sys/module/qla2xxx/parameters/ql2xmaxqdepth

  3. Verify that the queue depth values are updated:

    cat /sys/module/qla2xxx/parameters/ql2xmaxqdepth

    The current value of the queue depth is displayed.

  4. Modify the QLogic HBA queue depth by updating the firmware parameter Execution Throttle from the QLogic HBA BIOS.

    1. Log in to the QLogic HBA management CLI:

      /opt/QLogic_Corporation/QConvergeConsoleCLI/qaucli

    2. From the main menu, select the Adapter Configuration option.

      [root@localhost ~]# /opt/QLogic_Corporation/QConvergeConsoleCLI/qaucli
Using config file: /opt/QLogic_Corporation/QConvergeConsoleCLI/qaucli.cfg
Installation directory: /opt/QLogic_Corporation/QConvergeConsoleCLI
Working dir: /root

QConvergeConsole

        CLI - Version 2.2.0 (Build 15)

    Main Menu

    1:  Adapter Information
    **2:  Adapter Configuration**
    3:  Adapter Updates
    4:  Adapter Diagnostics
    5:  Monitoring
    6:  FabricCache CLI
    7:  Refresh
    8:  Help
    9:  Exit


        Please Enter Selection: 2

    3. From the list of adapter configuration parameters, select the HBA Parameters option.

      1:  Adapter Alias
    2:  Adapter Port Alias
    **3:  HBA Parameters**
    4:  Persistent Names (udev)
    5:  Boot Devices Configuration
    6:  Virtual Ports (NPIV)
    7:  Target Link Speed (iiDMA)
    8:  Export (Save) Configuration
    9:  Generate Reports
   10:  Personality
   11:  FEC
(p or 0: Previous Menu; m or 98: Main Menu; ex or 99: Quit)
        Please Enter Selection: 3

    4. From the list of HBA ports, select the required HBA port.

      Fibre Channel Adapter Configuration

    HBA Model QLE2562 SN: BFD1524C78510
      1: Port   1: WWPN: 21-00-00-24-FF-8D-98-E0 Online
      2: Port   2: WWPN: 21-00-00-24-FF-8D-98-E1 Online
    HBA Model QLE2672 SN: RFE1241G81915
      3: Port   1: WWPN: 21-00-00-0E-1E-09-B7-62 Online
      4: Port   2: WWPN: 21-00-00-0E-1E-09-B7-63 Online


        (p or 0: Previous Menu; m or 98: Main Menu; ex or 99: Quit)
        Please Enter Selection: 1

      The details of the HBA port are displayed.

    5. From the HBA Parameters menu, select the Display HBA Parameters option to view the current value of the Execution Throttle option.

      The default value of the Execution Throttle option is 65535.

      HBA Parameters Menu

=======================================================
HBA           : 2 Port: 1
SN            : BFD1524C78510
HBA Model     : QLE2562
HBA Desc.     : QLE2562 PCI Express to 8Gb FC Dual Channel
FW Version    : 8.01.02
WWPN          : 21-00-00-24-FF-8D-98-E0
WWNN          : 20-00-00-24-FF-8D-98-E0
Link          : Online
=======================================================

    1:  Display HBA Parameters
    2:  Configure HBA Parameters
    3:  Restore Defaults


        (p or 0: Previous Menu; m or 98: Main Menu; x or 99: Quit)
        Please Enter Selection: 1
--------------------------------------------------------------------------------
HBA Instance 2: QLE2562 Port 1 WWPN 21-00-00-24-FF-8D-98-E0 PortID 03-07-00
Link: Online
--------------------------------------------------------------------------------
Connection Options             : 2 - Loop Preferred, Otherwise Point-to-Point
Data Rate                      : Auto
Frame Size                     : 2048
Hard Loop ID                   : 0
Loop Reset Delay (seconds)     : 5
Enable Host HBA BIOS           : Enabled
Enable Hard Loop ID            : Disabled
Enable FC Tape Support         : Enabled
Operation Mode                 : 0 - Interrupt for every I/O completion
Interrupt Delay Timer (100us)  : 0
**Execution Throttle             : 65535**
Login Retry Count              : 8
Port Down Retry Count          : 30
Enable LIP Full Login          : Enabled
Link Down Timeout (seconds)    : 30
Enable Target Reset            : Enabled
LUNs Per Target                : 128
Out Of Order Frame Assembly    : Disabled
Enable LR Ext. Credits         : Disabled
Enable Fabric Assigned WWN     : N/A

Press <Enter> to continue:

    6. Press Enter to continue.

    7. From the HBA Parameters menu, select the Configure HBA Parameters option to modify the HBA parameters.

    8. From the Configure Parameters menu, select the Execute Throttle option and update the value of this parameter.

      Configure Parameters Menu

=======================================================
HBA           : 2 Port: 1
SN            : BFD1524C78510
HBA Model     : QLE2562
HBA Desc.     : QLE2562 PCI Express to 8Gb FC Dual Channel
FW Version    : 8.01.02
WWPN          : 21-00-00-24-FF-8D-98-E0
WWNN          : 20-00-00-24-FF-8D-98-E0
Link          : Online
=======================================================

    1:  Connection Options
    2:  Data Rate
    3:  Frame Size
    4:  Enable HBA Hard Loop ID
    5:  Hard Loop ID
    6:  Loop Reset Delay (seconds)
    7:  Enable BIOS
    8:  Enable Fibre Channel Tape Support
    9:  Operation Mode
   10:  Interrupt Delay Timer (100 microseconds)
   11:  Execution Throttle
   12:  Login Retry Count
   13:  Port Down Retry Count
   14:  Enable LIP Full Login
   15:  Link Down Timeout (seconds)
   16:  Enable Target Reset
   17:  LUNs per Target
   18:  Enable Receive Out Of Order Frame
   19:  Enable LR Ext. Credits
   20:  Commit Changes
   21:  Abort Changes


        (p or 0: Previous Menu; m or 98: Main Menu; x or 99: Quit)
        Please Enter Selection: 11
Enter Execution Throttle [1-65535] [65535]: 65500

    9. Press Enter to continue.

    10. From the Configure Parameters menu, select the Commit Changes option to save the changes.

    11. Exit the menu.