English

Upgrade compute node firmware

Contributors netapp-dbagwell netapp-amitha netapp-mwallis Download PDF of this page

For H-series compute nodes, you can upgrade the firmware for hardware components such as the BMC, BIOS, and NIC. To upgrade compute node firmware, you can use the NetApp Hybrid Cloud Control UI, REST API, a USB drive with the latest firmware image, or the BMC UI.

After the upgrade, the compute node boots into ESXi and works as before, retaining the configuration.

What you’ll need
  • Compute drivers: You have upgraded your compute node drivers.

  • Admin privileges: You have cluster administrator and BMC administrator permissions to perform the upgrade.

  • System ports: If you are using NetApp Hybrid Cloud Control for upgrades, you have ensured that the necessary ports are open. See Network ports for more information.

  • Minimum BMC and BIOS versions: The node you intend to upgrade using NetApp Hybrid Cloud Control meets the following minimum requirements:

    Model Minimum BMC version Minimum BIOS version

    H300E, H500E, H700E​

    6.84.00

    NA2.1

    H410C​

    All versions supported (no upgrade required)​

    All versions supported (no upgrade required)​

    H610C​

    3.96.07​

    3B01

    H615C​

    4.68.07

    3B08.CO ​ ​

    H615C compute nodes must update BMC firmware to version 4.68 using the compute firmware bundle 2.27 to enable NetApp Hybrid Cloud Control to perform future firmware upgrades.
    A Redfish license is required for H300E, H500E, and H700E compute nodes to enable NetApp Hybrid Cloud Control to perform future firmware upgrades. Contact NetApp Support to get the license installed manually until a new compute firmware bundle automates this process in a follow-on release.
    For a complete matrix of firmware and driver firmware for your hardware, see this KB article (login required).
  • BIOS boot order: Manually change the boot order in the BIOS setup for each node to ensure USB CD/DVD appears in the boot list. See this article for more information.

  • BMC credentials: Update your BMC credentials for your node using either the NetApp Hybrid Cloud Control UI or API methods. Updating BMC information prior to upgrade refreshes the inventory and ensures that management node services are aware of all hardware parameters needed to complete the upgrade.

  • Attached media: Disconnect any physical USB or ISO before starting a compute node upgrade.

  • KVM ESXi console: Close all open Serial-Over-LAN (SOL) sessions and active KVM sessions in the BMC UI before starting a compute node upgrade.

  • Witness Node requirements: In two- and three-node storage clusters, one Witness Node must be running in the NetApp HCI installation at all times.

  • Compute node health check: You have verified that the node is ready to be upgraded. See Run compute node health checks prior to upgrading compute firmware.

About this task

In production environments, upgrade the firmware on one compute node at a time.

For NetApp Hybrid Cloud Control UI or API upgrades, your ESXi host will be automatically placed in maintenance mode during the upgrade process if you have the DRS feature and required licensing. The node will be rebooted and after the upgrade process is complete, the ESXi host will be taken out of maintenance mode. For USB and BMC UI options, you will need to place the ESXi host in maintenance mode manually, as described in each procedure.

Upgrade options

Choose the option that is relevant to your upgrade scenario:

Use NetApp Hybrid Cloud Control UI to upgrade a compute node

Starting with management services 2.14, you can upgrade a compute node using the NetApp Hybrid Cloud Control UI. From the list of nodes, you must select the node to upgrade. The Current Versions tab shows the current firmware versions and the Proposed Versions tab shows the available upgrade versions, if any.

For a successful upgrade, ensure that the health check on the vSphere cluster is successful.
For dark site upgrades, you can reduce upload time if the upgrade package and the management node are both local.
Upgrading the NIC, BIOS, and BMC can take approximately 60 minutes per node depending on the speed of network connectivity between the management node and the BMC host.
What you’ll need
  • If your management node is not connected to the internet, you have downloaded the package from the NetApp Support Site.

Steps
  1. Open a web browser and browse to the IP address of the management node:

    https://<ManagementNodeIP>
  2. Log in to NetApp Hybrid Cloud Control by providing the storage cluster administrator credentials.

  3. Click Upgrade near the top right of the interface.

  4. On the Upgrades page, select Compute firmware.

  5. Choose from the following options and perform the set of steps that are applicable to your cluster:

    Option Steps

    Your management node has external connectivity.

    1. Select the cluster you are upgrading.

      You will see the nodes in the cluster listed along with the current firmware versions and newer versions, if available for upgrade.

    2. Select the upgrade package.

    3. Click Begin Upgrade.

      After you click Begin Upgrade, the window shows failed health checks, if any.

      The upgrade cannot be paused after you begin. Firmware will be updated sequentially in the following order: NIC, BIOS, and BMC. Do not log in to the BMC UI during upgrade. Logging into the BMC terminates the Hybrid Cloud Control Serial-Over-LAN (SOL) session that monitors upgrade process.
    4. If the health checks at the cluster or node level passed with warnings, but without critical failures, you will see Ready to be Upgraded. Click Upgrade Node.

    While the upgrade is in progress, you can leave the page and come back to it later to continue monitoring the progress. During the upgrade, the UI shows various messages about the status of the upgrade.
    While upgrading the firmware on H610C compute nodes, do not open the Serial-Over-LAN (SOL) console through the BMC web UI. This might cause the upgrade to fail.

    The UI displays a message after the upgrade is complete. You can download logs after the upgrade is complete.

    Your management node is within a dark site without external connectivity.

    1. Select the cluster you are upgrading.

    2. Click Browse to upload the upgrade package that you downloaded from the
      NetApp Support Site.

    3. Wait for the upload to complete. A progress bar shows the status of the upload.

    The file upload will happen in the background if you navigate away from the browser window.

    An on-screen message is displayed after the file is successfully uploaded and validated. Validation might take several minutes.
    You can download logs after the upgrade is complete. For information about the various upgrade status changes, see Upgrade status changes.

If a failure happens during the upgrade, NetApp Hybrid Cloud Control will reboot the node, take it out of maintenance mode, and display the failure status with a link to the error log. You can download the error log, which contains specific instructions or links to KB articles, to diagnose and correct any issue. For additional insight into compute node firmware upgrade issues using NetApp Hybrid Cloud Control, see this KB article.

Upgrade status changes

Here are the different states that the UI shows before, during, and after the upgrade process:

Upgrade state Description

Node failed one or more health checks. Expand to view details.

One or more health checks failed.

Error

An error has occurred during the upgrade. You can download the error log and send it to NetApp Support.

Unable to Detect

NetApp Hybrid Cloud Control does not have external connectivity to reach the online software repository. This status is also displayed if NetApp Hybrid Cloud Control is unable to query the compute node when the compute node asset does not have the hardware tag.

Ready to be Upgraded.

All the health checks passed successfully, and the node is ready to be upgraded.

An error has occurred during the upgrade.

The upgrade fails with this notification when a critical error occurs. Download the logs by clicking the Download Logs link to help resolve the error. You can try upgrading again after you resolve the error.

Node upgrade is in progress.

The upgrade is in progress. A progress bar shows the upgrade status.

Use NetApp Hybrid Cloud Control API to upgrade a compute node

You can use APIs to upgrade each compute node in a cluster to the latest firmware version. You can use an automation tool of your choice to run the APIs. The API workflow documented here uses the REST API UI available on the management node as an example.

What you’ll need

Compute node assets, including vCenter and hardware assets, must be known to management node assets. You can use the inventory service APIs to verify assets (https://[management node IP]/inventory/1/).

Steps
  1. Do one of the following depending on your connection:

    Option Steps

    Your management node has external connectivity.

    1. Verify the repository connection:

      1. Open the package service REST API UI on the management node:

        https://[management node IP]/package-repository/1/
      2. Click Authorize and complete the following:

        1. Enter the cluster user name and password.

        2. Enter the client ID as mnode-client.

        3. Click Authorize to begin a session.

        4. Close the authorization window.

      3. From the REST API UI, click GET ​/packages​/remote-repository​/connection.

      4. Click Try it out.

      5. Click Execute.

      6. If code 200 is returned, go to the next step. If there is no connection to the remote repository, establish the connection or use the dark site option.

    2. Find the upgrade package ID:

      1. From the REST API UI, click GET /packages.

      2. Click Try it out.

      3. Click Execute.

      4. From the response, copy and save the upgrade package name ("packageName") and package version ("packageVersion") for use in a later step.

    Your management node is within a dark site without external connectivity.

    1. Go to the NetApp HCI software download page and download the latest compute node firmware image to a device that is accessible to the management node.

      For dark site upgrades, you can reduce upload time if the upgrade package and the management node are both local.
    2. Upload the compute firmware upgrade package to the management node:

      1. Open the management node REST API UI on the management node:

        https://[management node IP]/package-repository/1/
      2. Click Authorize and complete the following:

        1. Enter the cluster user name and password.

        2. Enter the client ID as mnode-client.

        3. Click Authorize to begin a session.

        4. Close the authorization window.

      3. From the REST API UI, click POST /packages.

      4. Click Try it out.

      5. Click Browse and select the upgrade package.

      6. Click Execute to initiate the upload.

      7. From the response, copy and save the package ID ("id") for use in a later step.

    3. Verify the status of the upload.

      1. From the REST API UI, click GET​ /packages​/{id}​/status.

      2. Click Try it out.

      3. Enter the package ID you copied in the previous step in id.

      4. Click Execute to initiate the status request.

        The response indicates state as finished when complete.

      5. From the response, copy and save the upgrade package name ("name") and package version ("version") for use in a later step.

  2. Locate the compute controller ID and node hardware ID for the node you intend to upgrade:

    1. Open the inventory service REST API UI on the management node:

      https://[management node IP]/inventory/1/
    2. Click Authorize and complete the following:

      1. Enter the cluster user name and password.

      2. Enter the client ID as mnode-client.

      3. Click Authorize to begin a session.

      4. Close the authorization window.

    3. From the REST API UI, click GET /installations.

    4. Click Try it out.

    5. Click Execute.

    6. From the response, copy the installation asset ID ("id").

    7. From the REST API UI, click GET /installations/{id}.

    8. Click Try it out.

    9. Paste the installation asset ID into the id field.

    10. Click Execute.

    11. From the response, copy and save the cluster controller ID ("controllerId")and node hardware ID ("hardwareId") for use in a later step:

      "compute": {
        "errors": [],
        "inventory": {
          "clusters": [
            {
              "clusterId": "Test-1B",
              "controllerId": "a1b23456-c1d2-11e1-1234-a12bcdef123a",
      "nodes": [
        {
          "bmcDetails": {
            "bmcAddress": "10.111.0.111",
            "credentialsAvailable": true,
            "credentialsValidated": true
          },
          "chassisSerialNumber": "111930011231",
          "chassisSlot": "D",
          "hardwareId": "123a4567-01b1-1243-a12b-11ab11ab0a15",
          "hardwareTag": "00000000-0000-0000-0000-ab1c2de34f5g",
          "id": "e1111d10-1a1a-12d7-1a23-ab1cde23456f",
          "model": "H410C",
  3. Run the compute node firmware upgrade:

    1. Open the hardware service REST API UI on the management node:

      https://[management node IP]/hardware/2/
    2. Click Authorize and complete the following:

      1. Enter the cluster user name and password.

      2. Enter the client ID as mnode-client.

      3. Click Authorize to begin a session.

      4. Close the authorization window.

    3. Click POST /nodes/{hardware_id}/upgrades.

    4. Click Try it out.

    5. Enter the hardware host asset ID ("hardwareId" saved from a previous step) in the parameter field.

    6. Do the following with the payload values:

      1. Retain the values "force": false and "maintenanceMode": true" so that health checks are performed on the node and the ESXi host is set to maintenance mode.

      2. Enter the cluster controller ID ("controllerId" saved from a previous step).

      3. Enter the package name and package version you saved from a previous step.

        {
          "config": {
            "force": false,
            "maintenanceMode": true
          },
          "controllerId": "a1b23456-c1d2-11e1-1234-a12bcdef123a",
          "packageName": "compute-firmware-12.2.109",
          "packageVersion": "12.2.109"
        }
    7. Click Execute to initiate the upgrade.

      The upgrade cannot be paused after you begin. Firmware will be updated sequentially in the following order: NIC, BIOS, and BMC. Do not log in to the BMC UI during upgrade. Logging into the BMC terminates the Hybrid Cloud Control Serial-Over-LAN (SOL) session that monitors upgrade process.
    8. Copy the upgrade task ID that is part of the resource link ("resourceLink") URL in the response.

  4. Verify the upgrade progress and results:

    1. Click GET /task/{task_id}/logs.

    2. Click Try it out.

    3. Enter the task ID from the previous step in task_Id.

    4. Click Execute.

    5. Do one of the following if there are problems or special requirements during the upgrade:

      Option Steps

      You need to correct cluster health issues due to failedHealthChecks message in the response body.

      1. Go to the specific KB article listed for each issue or perform the specified remedy.

      2. If a KB is specified, complete the process described in the relevant KB article.

      3. After you have resolved cluster issues, reauthenticate if needed and click POST /nodes/{hardware_id}/upgrades.

      4. Repeat the steps as described previously in the upgrade step.

      The upgrade fails and the mitigation steps are not listed in upgrade log.

    6. Run the GET ​/task/{task_id}/logs API multiple times, as needed, until the process is complete.

      During the upgrade, the status indicates running if no errors are encountered. As each step finishes, the status value changes to completed.

      The upgrade has finished successfully when the status for each step is completed and the percentageCompleted value is 100.

  5. (Optional) Confirm upgraded firmware versions for each component:

    1. Open the hardware service REST API UI on the management node:

      https://[management node IP]/hardware/2/
    2. Click Authorize and complete the following:

      1. Enter the cluster user name and password.

      2. Enter the client ID as mnode-client.

      3. Click Authorize to begin a session.

      4. Close the authorization window.

    3. From the REST API UI, click GET ​/nodes​/{hardware_id}​/upgrades.

    4. (Optional) Enter date and status parameters to filter the results.

    5. Enter the hardware host asset ID ("hardwareId" saved from a previous step) in the parameter field.

    6. Click Try it out.

    7. Click Execute.

    8. Verify in the response that firmware for all components has been successfully upgraded from the previous version to the latest firmware.

Use a USB drive with the latest firmware image downloaded

You can insert a USB drive with the latest firmware image downloaded into a USB port on the compute node. As an alternative to using the USB thumb drive method described in this procedure, you can mount the compute node RTFI image on the compute node using the Virtual CD/DVD option in the Virtual Console in the Baseboard Management Controller (BMC) interface. The BMC method takes considerably longer than the USB thumb drive method. Ensure that your workstation or server has the necessary network bandwidth and that your browser session with the BMC does not time out.

Steps
  1. Browse to the NetApp software downloads page, click NetApp HCI, and click the download link for correct version of NetApp HCI.

  2. Accept the End User License Agreement.

  3. Under the Compute and Storage Nodes section, download the compute node image.

  4. Write the raw contents of the compute node RTFI image to a USB thumb drive with at least 32GB capacity (using dd or Etcher).

  5. Place the compute node in maintenance mode using VMware vCenter, and evacuate all virtual machines from the host.

    If VMware Distributed Resource Scheduler (DRS) is enabled on the cluster (this is the default in NetApp HCI installations), virtual machines will automatically be migrated to other nodes in the cluster.
  6. Insert the USB thumb drive into a USB port on the compute node and reboot the compute node using VMware vCenter.

  7. During the compute node POST cycle, press F11 to open the Boot Manager. You may need to press F11 multiple times in quick succession. You can perform this operation by connecting a video/keyboard or by using the console in BMC.

  8. Select One Shot > USB Flash Drive from the menu that appears. If the USB thumb drive does not appear in the menu, verify that USB Flash Drive is part of the legacy boot order in the BIOS of the system.

  9. Press Enter to boot the system from the USB thumb drive. The firmware flash process begins.

    After firmware flashing is complete and the node reboots, it might take a few minutes for ESXi to start.

  10. After the reboot is complete, exit maintenance mode on the upgraded compute node using vCenter.

  11. Remove the USB flash drive from the upgraded compute node.

  12. Repeat this task for other compute nodes in your ESXi cluster until all compute nodes are upgraded.

Use the Baseboard Management Controller (BMC) user interface (UI)

You must perform the sequential steps to load the compute firmware ISO and reboot the node to the ISO to ensure that the upgrade is successful. The ISO should be located on the system or virtual machine (VM) hosting the web browser. Ensure that you have downloaded the ISO before you start the process.

The recommendation is to have the system or VM and the node on the same network.
It takes approximately 25 to 30 minutes for the upgrade via the BMC UI.

Upgrade firmware on H410C and H300E/H500E/H700E nodes

If your node is part of a cluster, you must place the node in maintenance mode before the upgrade, and take it out of maintenance mode after the upgrade.

Ignore the following informational message you see during the process: Untrusty Debug Firmware Key is used, SecureFlash is currently in Debug Mode
Steps
  1. If your node is part of a cluster, place it in maintenance mode as follows. If not, skip to step 2.

    1. Log in to the VMware vCenter web client.

    2. Right-click the host (compute node) name and select Maintenance Mode > Enter Maintenance Mode.

    3. Click OK.
      VMs on the host will be migrated to another available host. VM migration can take time depending on the number of VMs that need to be migrated.

      Ensure that all the VMs on the host are migrated before you proceed.
  2. Navigate to the BMC UI, https://BMCIP/#login, where BMCIP is the IP address of the BMC.

  3. Log in using your credentials.

  4. Select Remote Control > Console Redirection.

  5. Click Launch Console.

    You might have to install Java or update it.
  6. When the console opens, click Virtual Media > Virtual Storage.

  7. On the Virtual Storage screen, click Logical Drive Type, and select ISO File.

    Shows the navigation path to select the ISO file.

  8. Click Open Image to browse to the folder where you downloaded the ISO file, and select the ISO file.

  9. Click Plug In.

  10. When the connection status shows Device#: VM Plug-in OK!!, click OK.

  11. Reboot the node by pressing F12 and clicking Restart or clicking Power Control > Set Power Reset.

  12. During reboot, press F11 to select the boot options and load the ISO. You might have to press F11 a few times before the boot menu is displayed.

    You will see the following screen:

    Shows the screen the virtual ISO boots up to.

  13. On the above screen, press Enter. Depending on your network, it might take a few minutes after you press Enter for the upgrade to begin.

    NOTE: Some of the firmware upgrades might cause the console to disconnect and/or cause your session on the BMC to disconnect. You can log back into the BMC, however some services, such as the console, may not be available due to the firmware upgrades. After the upgrades have completed, the node will perform a cold reboot, which can take approximately five minutes.
  14. Log back in to the BMC UI and click System to verify the BIOS version and build time after booting to the OS. If the upgrade completed correctly, you see the new BIOS and BMC versions.

    The BIOS version will not show the upgraded version until the node has finished fully booting.
  15. If the node is part of a cluster, complete the steps below. If it is a standalone node, no further action is needed.

    1. Log in to the VMware vCenter web client.

    2. Take the host out of maintenance mode. This might show a disconnected red flag. Wait until all statuses are cleared.

    3. Power on any of the remaining VMs that were powered off.

Upgrade firmware on H610C/H615C nodes

The steps vary depending on whether the node is standalone or part of a cluster. The procedure can take approximately 25 minutes and includes powering the node off, uploading the ISO, flashing the devices, and powering the node back on after the upgrade.

Steps
  1. If your node is part of a cluster, place it in maintenance mode as follows. If not, skip to step 2.

    1. Log in to the VMware vCenter web client.

    2. Right-click the host (compute node) name and select Maintenance Mode > Enter Maintenance Mode.

    3. Click OK.
      VMs on the host will be migrated to another available host. VM migration can take time depending on the number of VMs that need to be migrated.

      Ensure that all the VMs on the host are migrated before you proceed.
  2. Navigate to the BMC UI, https://BMCIP/#login, where BMC IP is the IP address of the BMC.

  3. Log in using your credentials.

  4. Click Remote Control > Launch KVM (Java).

  5. In the console window, click Media > Virtual Media Wizard.

    Start the Virtual Media Wizard from the BMC UI.
  6. Click Browse and select the compute firmware .iso file.

  7. Click Connect.
    A popup indicating success is displayed, along with the path and device showing at the bottom. You can close the Virtual Media window.

    Popup window showing ISO upload success.
  8. Reboot the node by pressing F12 and clicking Restart or clicking Power Control > Set Power Reset.

  9. During reboot, press F11 to select the boot options and load the ISO.

  10. Select AMI Virtual CDROM from the list displayed and click Enter. If you do not see AMI Virtual CDROM in the list, go into the BIOS and enable it in the boot list. The node will reboot after you save. During the reboot, press F11.

    Shows the window where you can select the boot device.
  11. On the screen displayed, click Enter.

    Some of the firmware upgrades might cause the console to disconnect and/or cause your session on the BMC to disconnect. You can log back into the BMC, however some services, such as the console, might not be available due to the firmware upgrades. After the upgrades have completed, the node will perform a cold reboot, which can take approximately five minutes.
  12. If you get disconnected from the console, select Remote Control and click Launch KVM or Launch KVM (Java) to reconnect and verify when the node has finished booting back up. You might need multiple reconnects to verify that the node booted successfully.

    During the powering on process, for approximately five minutes, the KVM console displays No Signal.
  13. After the node is powered on, select Dashboard > Device Information > More info to verify the BIOS and BMC versions. The upgraded BIOS and BMC versions are displayed. The upgraded version of the BIOS will not be displayed until the node has fully booted up.

  14. If you placed the node in maintenance mode, after the node boots to ESXi, right-click the host (compute node) name, and select Maintenance Mode > Exit Maintenance Mode, and migrate the VMs back to the host.

  15. In vCenter, with the host name selected, configure and verify the BIOS version.

Find more information