Installing GPU drivers for GPU-enabled compute nodes

Compute nodes with NVIDIA graphics processing units (GPUs), like the NetApp H610C, need the NVIDIA software drivers installed in VMware ESXi so that they can take advantage of the increased processing power. After deploying compute nodes with GPUs, you need to perform these steps on each GPU-enabled compute node to install the GPU drivers in ESXi.

Steps

  1. Open a browser and browse to the NVIDIA licensing portal at the following URL:
    https://nvid.nvidia.com/dashboard/
  2. Download one of the following driver packages to your computer, depending on your environment:
    vSphere version Driver package
    vSphere 6.0 NVIDIA-GRID-vSphere-6.0-390.94-390.96-392.05.zip
    vSphere 6.5 NVIDIA-GRID-vSphere-6.5-410.92-410.91-412.16.zip
    vSphere 6.7 NVIDIA-GRID-vSphere-6.7-410.92-410.91-412.16.zip
  3. Extract the driver package on your computer.
    The resulting .VIB file is the uncompressed driver file.
  4. Copy the .VIB driver file from your computer to ESXi running on the compute node. The following example commands for each version assume that the driver is located in the $HOME/NVIDIA/ESX6.x/ directory on the management host. The SCP utility is readily available in most Linux distributions, or available as a downloadable utility for all versions of Windows:
    Option Description
    ESXi 6.0 scp $HOME/NVIDIA/ESX6.0/NVIDIA**.vib root@<ESXi_IP_ADDR>:/.
    ESXi 6.5 scp $HOME/NVIDIA/ESX6.5/NVIDIA**.vib root@<ESXi_IP_ADDR>:/.
    ESXi 6.7 scp $HOME/NVIDIA/ESX6.7/NVIDIA**.vib root@<ESXi_IP_ADDR>:/.
  5. Use the following steps to log in as root to the ESXi host and install the NVIDIA vGPU manager in ESXi.
    1. Run the following command to log in to the ESXi host as the root user:
      ssh root@<ESXi_IP_ADDRESS>
    2. Run the following command to verify that no NVIDIA GPU drivers are currently installed:
      nvidia-smi
      This command should return the message "nvidia-smi: not found".
    3. Run the following commands to enable maintenance mode on the host and install the NVIDIA vGPU Manager from the VIB file:
      esxcli system maintenanceMode set --enable true
      esxcli software vib install -v /NVIDIA**.vib
      You should see the message "Operation finished successfully".
    4. Run the following command and verify that all eight GPU drivers are listed in the command output:
      nvidia-smi
    5. Run the following command to verify that the NVIDIA vGPU package was installed and loaded correctly:
      vmkload_mod -l | grep nvidia
      The command should return output similar to the following: nvidia 816 13808
    6. Run the following command to reboot the host:
      reboot -f
    7. Run the following command to exit maintenance mode:
      esxcli system maintenanceMode set --enable false
  6. Repeat steps 4-6 for any other newly deployed compute nodes with NVIDIA GPUs.
  7. Perform the following tasks using the instructions in the NVIDIA documentation site:
    1. Install the NVIDIA license server.
    2. Configure the virtual machine guests for NVIDIA vGPU software.
    3. If you are using vGPU-enabled desktops in a virtual desktop infrastructure (VDI) context, configure VMware Horizon View for NVIDIA vGPU software.