Replace the NVMEM battery - AFF A220 and FAS2700

Contributors dougthomp netapp-martyh thrisun

To replace an NVMEM battery in the system, you must remove the controller module from the system, open it, replace the battery, and close and replace the controller module.

All other components in the system must be functioning properly; if not, you must contact technical support.

Step 1: Shut down the impaired controller

You can shut down or take over the impaired controller using different procedures, depending on the storage system hardware configuration.

Option 1: Most configurations

To shut down the impaired node, you must determine the status of the node and, if necessary, take over the node so that the healthy node continues to serve data from the impaired node storage.

About this task

If you have a cluster with more than two nodes, it must be in quorum. If the cluster is not in quorum or a healthy node shows false for eligibility and health, you must correct the issue before shutting down the impaired node; see the Administration overview with the CLI.

Steps
  1. If AutoSupport is enabled, suppress automatic case creation by invoking an AutoSupport message: system node autosupport invoke -node * -type all -message MAINT=number_of_hours_downh

    The following AutoSupport message suppresses automatic case creation for two hours: cluster1:*> system node autosupport invoke -node * -type all -message MAINT=2h

  2. Disable automatic giveback from the console of the healthy node: storage failover modify –node local -auto-giveback false

  3. Take the impaired node to the LOADER prompt:

    If the impaired node is displaying…​ Then…​

    The LOADER prompt

    Go to the next step.

    Waiting for giveback…​

    Press Ctrl-C, and then respond y when prompted.

    System prompt or password prompt (enter system password)

    Take over or halt the impaired node:

    • For an HA pair, take over the impaired node from the healthy node: storage failover takeover -ofnode impaired_node_name

      When the impaired node shows Waiting for giveback…​, press Ctrl-C, and then respond y.

  4. If the impaired node is part of an HA pair, disable automatic giveback from the console of the healthy node: storage failover modify -node local -auto-giveback false

  5. Take the impaired node to the LOADER prompt:

    If the impaired node is displaying…​ Then…​

    The LOADER prompt

    Go to the next step.

    Waiting for giveback…​

    Press Ctrl-C, and then respond y when prompted.

    System prompt or password prompt (enter system password)

    Take over or halt the impaired node:

    • For an HA pair, take over the impaired node from the healthy node: storage failover takeover -ofnode impaired_node_name

      When the impaired node shows Waiting for giveback…​, press Ctrl-C, and then respond y.

    • For a stand-alone system: system node halt impaired_node_name

  6. If the system has only one controller module in the chassis, turn off the power supplies, and then unplug the impaired node’s power cords from the power source.

Option 2: Controller is in a MetroCluster

Note Do not use this procedure if your system is in a two-node MetroCluster configuration.

To shut down the impaired node, you must determine the status of the node and, if necessary, take over the node so that the healthy node continues to serve data from the impaired node storage.

  • If you have a cluster with more than two nodes, it must be in quorum. If the cluster is not in quorum or a healthy node shows false for eligibility and health, you must correct the issue before shutting down the impaired node; see the Administration overview with the CLI.

  • If you have a MetroCluster configuration, you must have confirmed that the MetroCluster Configuration State is configured and that the nodes are in an enabled and normal state (metrocluster node show).

Steps
  1. If AutoSupport is enabled, suppress automatic case creation by invoking an AutoSupport message: system node autosupport invoke -node * -type all -message MAINT=number_of_hours_downh

    The following AutoSupport message suppresses automatic case creation for two hours: cluster1:*> system node autosupport invoke -node * -type all -message MAINT=2h

  2. Disable automatic giveback from the console of the healthy node: storage failover modify –node local -auto-giveback false

  3. Take the impaired node to the LOADER prompt:

    If the impaired node is displaying…​ Then…​

    The LOADER prompt

    Go to the next step.

    Waiting for giveback…​

    Press Ctrl-C, and then respond y when prompted.

    System prompt or password prompt (enter system password)

    Take over or halt the impaired node:

    • For an HA pair, take over the impaired node from the healthy node: storage failover takeover -ofnode impaired_node_name

      When the impaired node shows Waiting for giveback…​, press Ctrl-C, and then respond y.

Step 2: Open the system

To access components inside the controller, you must first remove the controller module from the system and then remove the cover on the controller module.

Steps
  1. If you are not already grounded, properly ground yourself.

  2. Loosen the hook and loop strap binding the cables to the cable management device, and then unplug the system cables and SFPs (if needed) from the controller module, keeping track of where the cables were connected.

    Leave the cables in the cable management device so that when you reinstall the cable management device, the cables are organized.

  3. Remove and set aside the cable management devices from the left and right sides of the controller module.

    drw 25xx cable management arm
  4. Squeeze the latch on the cam handle until it releases, open the cam handle fully to release the controller module from the midplane, and then, using two hands, pull the controller module out of the chassis.

    drw 2240 x opening cam latch
  5. Turn the controller module over and place it on a flat, stable surface.

  6. Open the cover by sliding in the blue tabs to release the cover, and then swing the cover up and open.

Step 3: Replace the NVMEM battery

To replace the NVMEM battery in your system, you must remove the failed NVMEM battery from the system and replace it with a new NVMEM battery.

Steps
  1. If you are not already grounded, properly ground yourself.

  2. Check the NVMEM LED:

    • If your system is in an HA configuration, go to the next step.

    • If your system is in a stand-alone configuration, cleanly shut down the controller module, and then check the NVRAM LED identified by the NV icon.

      drw hw nvram icon
      Note The NVRAM LED blinks while destaging contents to the flash memory when you halt the system. After the destage is complete, the LED turns off.
      • If power is lost without a clean shutdown, the NVMEM LED flashes until the destage is complete, and then the LED turns off.

      • If the LED is on and power is on, unwritten data is stored on NVMEM.

        This typically occurs during an uncontrolled shutdown after ONTAP has successfully booted.

  3. Locate the NVMEM battery in the controller module.

    drw 2600 nvmem batt repl animated gif
  4. Locate the battery plug and squeeze the clip on the face of the battery plug to release the plug from the socket, and then unplug the battery cable from the socket.

  5. Remove the battery from the controller module and set it aside.

  6. Remove the replacement battery from its package.

  7. Loop the battery cable around the cable channel on the side of the battery holder.

  8. Position the battery pack by aligning the battery holder key ribs to the “V” notches on the sheet metal side wall.

  9. Slide the battery pack down along the sheet metal side wall until the support tabs on the side wall hook into the slots on the battery pack, and the battery pack latch engages and clicks into the opening on the side wall.

  10. Plug the battery plug back into the controller module.

Step 4: Reinstall the controller module

After you replace components in the controller module, reinstall it into the chassis.

Steps
  1. If you are not already grounded, properly ground yourself.

  2. If you have not already done so, replace the cover on the controller module.

  3. Align the end of the controller module with the opening in the chassis, and then gently push the controller module halfway into the system.

    Note Do not completely insert the controller module in the chassis until instructed to do so.
  4. Recable the system, as needed.

    If you removed the media converters (QSFPs or SFPs), remember to reinstall them if you are using fiber optic cables.

  5. Complete the reinstallation of the controller module:

    If your system is in…​ Then perform these steps…​

    An HA pair

    The controller module begins to boot as soon as it is fully seated in the chassis. Be prepared to interrupt the boot process.

    1. With the cam handle in the open position, firmly push the controller module in until it meets the midplane and is fully seated, and then close the cam handle to the locked position.

      Note Do not use excessive force when sliding the controller module into the chassis to avoid damaging the connectors.

      The controller begins to boot as soon as it is seated in the chassis.

    2. If you have not already done so, reinstall the cable management device.

    3. Bind the cables to the cable management device with the hook and loop strap.

    4. When you see the message Press Ctrl-C for Boot Menu, press Ctrl-C to interrupt the boot process.

      Note If you miss the prompt and the controller module boots to ONTAP, enter halt, and then at the LOADER prompt enter boot_ontap, press Ctrl-C when prompted, and then boot to Maintenance mode.
    5. Select the option to boot to Maintenance mode from the displayed menu.

    A stand-alone configuration

    1. With the cam handle in the open position, firmly push the controller module in until it meets the midplane and is fully seated, and then close the cam handle to the locked position.

      Note Do not use excessive force when sliding the controller module into the chassis to avoid damaging the connectors.
    2. If you have not already done so, reinstall the cable management device.

    3. Bind the cables to the cable management device with the hook and loop strap.

    4. Reconnect the power cables to the power supplies and to the power sources, turn on the power to start the boot process, and then press Ctrl-C after you see the Press Ctrl-C for Boot Menu message.

      Note If you miss the prompt and the controller module boots to ONTAP, enter halt, and then at the LOADER prompt enter boot_ontap, press Ctrl-C when prompted, and then boot to Maintenance mode.
    5. From the boot menu, select the option for Maintenance mode.

Step 5: Run system-level diagnostics

After installing a new NVMEM battery, you should run diagnostics.

Your system must be at the LOADER prompt to start System Level Diagnostics.

All commands in the diagnostic procedures are issued from the node where the component is being replaced.

Steps
  1. If the node to be serviced is not at the LOADER prompt, perform the following steps:

    1. Select the Maintenance mode option from the displayed menu.

    2. After the node boots to Maintenance mode, halt the node: halt

      After you issue the command, you should wait until the system stops at the LOADER prompt.

      Note During the boot process, you can safely respond y to prompts:
      • A prompt warning that when entering Maintenance mode in an HA configuration, you must ensure that the healthy node remains down.

  2. At the LOADER prompt, access the special drivers specifically designed for system-level diagnostics to function properly: boot_diags

    During the boot process, you can safely respond y to the prompts until the Maintenance mode prompt (*>) appears.

  3. Run diagnostics on the NVMEM memory: sldiag device run -dev nvmem

  4. Verify that no hardware problems resulted from the replacement of the NVMEM battery: sldiag device status -dev nvmem -long -state failed

    System-level diagnostics returns you to the prompt if there are no test failures, or lists the full status of failures resulting from testing the component.

  5. Proceed based on the result of the preceding step:

    If the system-level diagnostics tests…​ Then…​

    Were completed without any failures

    1. Clear the status logs: sldiag device clearstatus

    2. Verify that the log was cleared: sldiag device status

      The following default response is displayed:

      SLDIAG: No log messages are present.

    3. Exit Maintenance mode: halt

      The node displays the LOADER prompt.

    4. Boot the node from the LOADER prompt: bye

    5. Return the node to normal operation:

    If your node is in…​ Then…​

    An HA pair

    Perform a give back: storage failover giveback -ofnode replacement_node_name

    Note If you disabled automatic giveback, re-enable it with the storage failover modify command.

    A two-node MetroCluster configuration

    Proceed to the next step.

    The MetroCluster switchback procedure is done in the next task in the replacement process.

    A stand-alone configuration

    Proceed to the next step.

    No action is required.

    You have completed system-level diagnostics.

    Resulted in some test failures

    Determine the cause of the problem:

    1. Exit Maintenance mode: halt

      After you issue the command, wait until the system stops at the LOADER prompt.

    2. Turn off or leave on the power supplies, depending on how many controller modules are in the chassis:

      • If you have two controller modules in the chassis, leave the power supplies turned on to provide power to the other controller module.

      • If you have one controller module in the chassis, turn off the power supplies and unplug them from the power sources.

    3. Verify that you have observed all the considerations identified for running system-level diagnostics, that cables are securely connected, and that hardware components are properly installed in the storage system.

    4. Boot the controller module you are servicing, interrupting the boot by pressing Ctrl-C when prompted to get to the Boot menu:

      • If you have two controller modules in the chassis, fully seat the controller module you are servicing in the chassis.

        The controller module boots up when fully seated.

      • If you have one controller module in the chassis, connect the power supplies, and then turn them on.

    5. Select Boot to maintenance mode from the menu.

    6. Exit Maintenance mode by entering the following command: halt

      After you issue the command, wait until the system stops at the LOADER prompt.

    7. Rerun the system-level diagnostic test.

Step 7: Switch back aggregates in a two-node MetroCluster configuration

After you have completed the FRU replacement in a two-node MetroCluster configuration, you can perform the MetroCluster switchback operation. This returns the configuration to its normal operating state, with the sync-source storage virtual machines (SVMs) on the formerly impaired site now active and serving data from the local disk pools.

This task only applies to two-node MetroCluster configurations.

Steps
  1. Verify that all nodes are in the enabled state: metrocluster node show

    cluster_B::>  metrocluster node show
    
    DR                           Configuration  DR
    Group Cluster Node           State          Mirroring Mode
    ----- ------- -------------- -------------- --------- --------------------
    1     cluster_A
                  controller_A_1 configured     enabled   heal roots completed
          cluster_B
                  controller_B_1 configured     enabled   waiting for switchback recovery
    2 entries were displayed.
  2. Verify that resynchronization is complete on all SVMs: metrocluster vserver show

  3. Verify that any automatic LIF migrations being performed by the healing operations were completed successfully: metrocluster check lif show

  4. Perform the switchback by using the metrocluster switchback command from any node in the surviving cluster.

  5. Verify that the switchback operation has completed: metrocluster show

    The switchback operation is still running when a cluster is in the waiting-for-switchback state:

    cluster_B::> metrocluster show
    Cluster              Configuration State    Mode
    --------------------	------------------- 	---------
     Local: cluster_B configured       	switchover
    Remote: cluster_A configured       	waiting-for-switchback

    The switchback operation is complete when the clusters are in the normal state.:

    cluster_B::> metrocluster show
    Cluster              Configuration State    Mode
    --------------------	------------------- 	---------
     Local: cluster_B configured      		normal
    Remote: cluster_A configured      		normal

    If a switchback is taking a long time to finish, you can check on the status of in-progress baselines by using the metrocluster config-replication resync-status show command.

  6. Reestablish any SnapMirror or SnapVault configurations.

Step 8: Return the failed part to NetApp

After you replace the part, you can return the failed part to NetApp, as described in the RMA instructions shipped with the kit. Contact technical support at NetApp Support, 888-463-8277 (North America), 00-800-44-638277 (Europe), or +800-800-80-800 (Asia/Pacific) if you need the RMA number or additional help with the replacement procedure.