Replace the NVRAM module and/or NVRAM DIMMs - AFF A700s

Contributors netapp-martyh thrisun dougthomp

To replace a failed NVRAM card, you must remove the NVRAM riser, Riser 1, from the controller module, remove the failed card from the riser, install the new NVRAM card in the riser, and then reinstall the riser in the controller module. Because the system ID is derived from the NVRAM card, if replacing the module, disks belonging to the system are reassigned to the new system ID.

Before you begin
  • All disk shelves must be working properly.

  • If your system is in an HA pair, the partner node must be able to take over the node associated with the NVRAM module that is being replaced.

  • This procedure uses the following terminology:

    • The impaired node is the node on which you are performing maintenance.

    • The healthy node is the HA partner of the impaired node.

  • This procedure includes steps for automatically or manually reassigning disks to the controller module associated with the new NVRAM module. You must reassign the disks when directed to in the procedure. Completing the disk reassignment before giveback can cause issues.

  • You must replace the failed component with a replacement FRU component you received from your provider.

  • You cannot change any disks or disk shelves as part of this procedure.

Shut down the impaired node

Steps

To shut down the impaired node, you must determine the status of the node and, if necessary, take over the node so that the healthy node continues to serve data from the impaired node storage.

About this task

If you have a cluster with more than two nodes, it must be in quorum. If the cluster is not in quorum or a healthy node shows false for eligibility and health, you must correct the issue before shutting down the impaired node; see the Administration overview with the CLI.

Steps
  1. If AutoSupport is enabled, suppress automatic case creation by invoking an AutoSupport message: system node autosupport invoke -node * -type all -message MAINT=number_of_hours_downh

    The following AutoSupport message suppresses automatic case creation for two hours: cluster1:*> system node autosupport invoke -node * -type all -message MAINT=2h

  2. Disable automatic giveback from the console of the healthy node: storage failover modify –node local -auto-giveback false

  3. Take the impaired node to the LOADER prompt:

    If the impaired node is displaying…​ Then…​

    The LOADER prompt

    Go to the next step.

    Waiting for giveback…​

    Press Ctrl-C, and then respond y when prompted.

    System prompt or password prompt (enter system password)

    Take over or halt the impaired node:

    • For an HA pair, take over the impaired node from the healthy node: storage failover takeover -ofnode impaired_node_name

      When the impaired node shows Waiting for giveback…​, press Ctrl-C, and then respond y.

    • If you are using NetApp Storage Encryption, you must have reset the MSID using the instructions in the “Returning SEDs to unprotected mode” section of the ONTAP 9 NetApp Encryption Power Guide.

Remove the controller module

You must remove the controller module from the chassis when you replace the controller module or replace a component inside the controller module.

  1. If you are not already grounded, properly ground yourself.

  2. Loosen the hook and loop strap binding the cables to the cable management device, and then unplug the system cables and SFPs (if needed) from the controller module, keeping track of where the cables were connected.

    Leave the cables in the cable management device so that when you reinstall the cable management device, the cables are organized.

  3. Unplug the controller module power supply from the source, and then unplug the cable from the power supply.

  4. Remove the cable management device from the controller module and set it aside.

  5. Press down on both of the locking latches, and then rotate both latches downward at the same time.

    The controller module moves slightly out of the chassis.

    drw a700s pcm remove

    legend icon 01

    Locking latch

    legend icon 02

    Locking pin

  6. Slide the controller module out of the chassis.

    Make sure that you support the bottom of the controller module as you slide it out of the chassis.

  7. Place the controller module on a stable, flat surface, and then open the air duct:

    1. Press in the locking tabs on the sides of the air duct toward the middle of the controller module.

    2. Slide the air duct toward the fan modules, and then rotate it upward to its completely open position.

      drw a700s open air duct

      legend icon 01

      Air duct locking tabs

      legend icon 02

      Risers

      legend icon 03

      Air duct

Remove the NVRAM card

Replacing the NVRAM consist of removing the NVRAM riser, Riser 1, from the controller module, disconnecting the NVRAM battery from the NVRAM card, removing the failed NVRAM card and installing the replacement NVRAM card, and then reinstalling the NVRAM riser back into the controller module.

  1. If you are not already grounded, properly ground yourself.

  2. Remove the NVRAM riser, Riser 1, from the controller module:

    1. Rotate the riser locking latch on the left side of the riser up and toward the fans.

      The NVRAM riser raises up slightly from the controller module.

    2. Lift the NVRAM riser up, shift it toward the fans so that the sheet metal lip on the riser clears the edge of the controller module, lift the riser straight up out of the controller module, and then place it on a stable, flat surface so that you can access the NVRAM card.

      drw a700s nvme replace

      legend icon 01

      Air duct

      legend icon 02

      Riser 1 locking latch

      legend icon 03

      NVRAM battery cable plug connecting to the NVRAM card

      legend icon 04

      Card locking bracket

      legend icon 05

      NVRAM card

  3. Remove the NVRAM card from the riser module:

    1. Turn the riser module so that you can access the NVRAM card.

    2. Unplug the NVRAM battery cable that is attached to the NVRAM card.

    3. Press the locking bracket on the side of the NVRAM riser, and then rotate it to the open position.

    4. Remove the NVRAM card from the riser module.

  4. Install the NVRAM card into the NVRAM riser:

    1. Align the card with the card guide on the riser module and the card socket in the riser.

    2. Slide the card squarely into the card socket.

      Note Make sure that the card is completely and squarely seated into the riser socket.
    3. Connect the battery cable to the socket on the NVRAM card.

    4. Swing the locking latch into the locked position and make sure that it locks in place.

  5. Install the riser into the controller module:

    1. Align the lip of the riser with the underside of the controller module sheet metal.

    2. Guide the riser along the pins in the controller module, and then lower the riser into the controller module.

    3. Swing the locking latch down and click it into the locked position.

      When locked, the locking latch is flush with the top of the riser and the riser sits squarely in the controller module.

    4. Reinsert any SFP modules that were removed from the PCIe cards.

Reinstall the controller module and booting the system

After you replace a FRU in the controller module, you must reinstall the controller module and reboot it.

For HA pairs with two controller modules in the same chassis, the sequence in which you install the controller module is especially important because it attempts to reboot as soon as you completely seat it in the chassis.

  1. If you are not already grounded, properly ground yourself.

  2. Align the end of the controller module with the opening in the chassis, and then gently push the controller module halfway into the system.

    Note Do not completely insert the controller module in the chassis until instructed to do so.
  3. Recable the system, as needed.

    If you removed the media converters (QSFPs or SFPs), remember to reinstall them if you are using fiber optic cables.

  4. Plug the power cord into the power supply, reinstall the power cable locking collar, and then connect the power supply to the power source.

  5. Complete the reinstallation of the controller module:

    1. If you have not already done so, reinstall the cable management device.

    2. Firmly push the controller module into the chassis until it meets the midplane and is fully seated.

      The locking latches rise when the controller module is fully seated.

      Note Do not use excessive force when sliding the controller module into the chassis to avoid damaging the connectors.

      The controller module begins to boot as soon as it is fully seated in the chassis. Be prepared to interrupt the boot process.

    3. Rotate the locking latches upward, tilting them so that they clear the locking pins, and then lower them into the locked position.

    4. Interrupt the boot process by pressing Ctrl-C when you see Press Ctrl-C for Boot Menu.

    5. Select the option to boot to Maintenance mode from the displayed menu.

Verify the system ID change on an HA system

You must confirm the system ID change when you boot the replacement node and then verify that the change was implemented.

This procedure applies only to systems running ONTAP in an HA pair.

  1. If the replacement node is in Maintenance mode (showing the *> prompt, exit Maintenance mode and go to the LOADER prompt: halt

  2. From the LOADER prompt on the replacement node, boot the node, entering y if you are prompted to override the system ID due to a system ID mismatch:boot_ontap

  3. Wait until the Waiting for giveback…​ message is displayed on the replacement node console and then, from the healthy node, verify that the new partner system ID has been automatically assigned: storage failover show

    In the command output, you should see a message that the system ID has changed on the impaired node, showing the correct old and new IDs. In the following example, node2 has undergone replacement and has a new system ID of 151759706.

    node1> storage failover show
                                        Takeover
    Node              Partner           Possible     State Description
    ------------      ------------      --------     -------------------------------------
    node1             node2             false        System ID changed on partner (Old:
                                                      151759755, New: 151759706), In takeover
    node2             node1             -            Waiting for giveback (HA mailboxes)
  4. From the healthy node, verify that any coredumps are saved:

    1. Change to the advanced privilege level: set -privilege advanced

      You can respond Y when prompted to continue into advanced mode. The advanced mode prompt appears (*>).

    2. Save any coredumps: system node run -node local-node-name partner savecore

    3. Wait for savecore command to complete before issuing the giveback.

      You can enter the following command to monitor the progress of the savecore command: system node run -node local-node-name partner savecore -s

    4. Return to the admin privilege level: set -privilege admin

  5. Give back the node:

    1. From the healthy node, give back the replaced node’s storage: storage failover giveback -ofnode replacement_node_name

      the replacement node takes back its storage and completes booting.

      If you are prompted to override the system ID due to a system ID mismatch, you should enter y.

      Note If the giveback is vetoed, you can consider overriding the vetoes.
    2. After the giveback has been completed, confirm that the HA pair is healthy and that takeover is possible: storage failover show

      The output from the storage failover show command. should not include the System ID changed on partner message.

  6. Verify that the disks were assigned correctly: storage disk show -ownership

    The disks belonging to the replacement node should show the new system ID. In the following example, the disks owned by node1 now show the new system ID, 1873775277:

    node1> storage disk show -ownership
    
    Disk  Aggregate Home  Owner  DR Home  Home ID    Owner ID  DR Home ID Reserver  Pool
    ----- ------    ----- ------ -------- -------    -------    -------  ---------  ---
    1.0.0  aggr0_1  node1 node1  -        1873775277 1873775277  -       1873775277 Pool0
    1.0.1  aggr0_1  node1 node1           1873775277 1873775277  -       1873775277 Pool0
    .
    .
    .
  7. Verify that the expected volumes are present for each node: vol show -node node-name

  8. If you disabled automatic takeover on reboot, enable it from the healthy node: storage failover modify -node replacement-node-name -onreboot true

Restore Storage and Volume Encryption functionality

After replacing the controller module or NVRAM module for a storage system that you previously configured to use Storage or Volume Encryption, you must perform additional steps to provide uninterrupted Encryption functionality. You can skip this task on storage systems that do not have Storage or Volume Encryption enabled.

Step
  1. Restore Storage or Volume Encryption functionality by using the appropriate procedure in NetApp Encryption overview with the CLI.

Use one of the following procedures, depending on whether you are using onboard or external key management:

  • “Restoring onboard key management encryption keys”

  • “Restoring external key management encryption keys”

Return the failed part to NetApp

After you replace the part, you can return the failed part to NetApp, as described in the RMA instructions shipped with the kit. Contact technical support at NetApp Support, 888-463-8277 (North America), 00-800-44-638277 (Europe), or +800-800-80-800 (Asia/Pacific) if you need the RMA number or additional help with the replacement procedure.