Replace a PCIe card - AFF A800

08/16/2023 Contributors

To replace a PCIe card, you must disconnect the cables from the cards, remove the SFP and QSFP modules from the cards before removing the riser, reinstall the riser, and then reinstall the SFP and QSFP modules before cabling the cards.

You can use this procedure with all versions of ONTAP supported by your system
All other components in the system must be functioning properly; if not, you must contact technical support.

Step 1: Shut down the impaired controller

To shut down the impaired controller, you must determine the status of the controller and, if necessary, take over the controller so that the healthy controller continues to serve data from the impaired controller storage.

About this task

If you have a SAN system, you must have checked event messages (cluster kernel-service show) for the impaired controller SCSI blade. The cluster kernel-service show command (from priv advanced mode) displays the node name, quorum status of that node, availability status of that node, and operational status of that node.

Each SCSI-blade process should be in quorum with the other nodes in the cluster. Any issues must be resolved before you proceed with the replacement.
If you have a cluster with more than two nodes, it must be in quorum. If the cluster is not in quorum or a healthy controller shows false for eligibility and health, you must correct the issue before shutting down the impaired controller; see Synchronize a node with the cluster.

Steps

If AutoSupport is enabled, suppress automatic case creation by invoking an AutoSupport message: system node autosupport invoke -node * -type all -message MAINT=<# of hours>h

The following AutoSupport message suppresses automatic case creation for two hours: cluster1:> system node autosupport invoke -node * -type all -message MAINT=2h
Disable automatic giveback from the console of the healthy controller: storage failover modify –node local -auto-giveback false

When you see Do you want to disable auto-giveback?, enter y.

Take the impaired controller to the LOADER prompt:

If the impaired controller is displaying… Then…

If the impaired controller is displaying…	Then…
The LOADER prompt	Go to the next step.
Waiting for giveback…	Press Ctrl-C, and then respond `y` when prompted.
System prompt or password prompt	Take over or halt the impaired controller from the healthy controller: `storage failover takeover -ofnode impaired_node_name` When the impaired controller shows Waiting for giveback…, press Ctrl-C, and then respond `y`.

The LOADER prompt

Go to the next step.

Waiting for giveback…

Press Ctrl-C, and then respond y when prompted.

System prompt or password prompt

Take over or halt the impaired controller from the healthy controller: storage failover takeover -ofnode impaired_node_name

When the impaired controller shows Waiting for giveback…, press Ctrl-C, and then respond y.

Step 2: Remove the controller module

You must remove the controller module from the chassis when you replace the controller module or replace a component inside the controller module.

If you are not already grounded, properly ground yourself.
Unplug the controller module power supplies from the source.
Release the power cable retainers, and then unplug the cables from the power supplies.
Loosen the hook and loop strap binding the cables to the cable management device, and then unplug the system cables and SFP and QSFP modules (if needed) from the controller module, keeping track of where the cables were connected.

Leave the cables in the cable management device so that when you reinstall the cable management device, the cables are organized.
Remove the cable management device from the controller module and set it aside.
Press down on both of the locking latches, and then rotate both latches downward at the same time.

The controller module moves slightly out of the chassis.

Locking latch

Locking pin
Slide the controller module out of the chassis.

Make sure that you support the bottom of the controller module as you slide it out of the chassis.
Place the controller module on a stable, flat surface, and then open the air duct:
1. Press in the locking tabs on the sides of the air duct toward the middle of the controller module.
2. Slide the air duct toward the fan modules, and then rotate it upward to its completely open position.
  
  Air duct locking tabs
  
  Slide air duct towards fan modules
  
  Rotate air duct towards fan modules

Step 3: Replace a PCIe card

To replace a PCIe card, you must remove the cabling and any QSFPs and SFPs from the ports on the PCIe cards in the target riser, remove the riser from the controller module, remove and replace the PCIe card, reinstall the riser and any QSFPs and SFPs onto the ports, and cable the ports.

Determine if the card you are replacing is from Riser 1 or if it is from Riser 2 or 3.
- If you are replacing the 100GbE PCIe card in Riser 1, use Steps 2 - 3 and Steps 6 - 7.
- If you are replacing a PCIe card from Riser 2 or 3, use Steps 4 through 7.
Remove Riser 1 from the controller module:
1. Remove the QSFP modules that might be in the PCIe card.
2. Rotate the riser locking latch on the left side of the riser up and toward the fan modules.
  
  The riser raises up slightly from the controller module.
3. Lift the riser up, shift it toward the fans so that the sheet metal lip on the riser clears the edge of the controller module, lift the riser out of the controller module, and then place it on a stable, flat surface.
  
  Air duct
  
  Riser locking latch
  
  Card locking bracket
  
  Riser 1 (left riser) with 100GbE PCIe card in slot 1.
Remove the PCIe card from Riser 1:
1. Turn the riser so that you can access the PCIe card.
2. Press the locking bracket on the side of the PCIe riser, and then rotate it to the open position.
3. Remove the PCIe card from the riser.
Remove the PCIe riser from the controller module:
1. Remove any SFP or QSFP modules that might be in the PCIe cards.
2. Rotate the riser locking latch on the left side of the riser up and toward the fan modules.
  
  The riser raises up slightly from the controller module.
3. Lift the riser up, shift it toward the fans so that the sheet metal lip on the riser clears the edge of the controller module, lift the riser out of the controller module, and then place it on a stable, flat surface.
  
  Air duct
  
  Riser 2 (middle riser) or 3 (right riser) locking latch
  
  Card locking bracket
  
  Side panel on riser 2 or 3
  
  PCIe cards in riser 2 or 3
Remove the PCIe card from the riser:
1. Turn the riser so that you can access the PCIe cards.
2. Press the locking bracket on the side of the PCIe riser, and then rotate it to the open position.
3. Swing the side panel off the riser.
4. Remove the PCIe card from the riser.
Install the PCIe card into the same slot in the riser:
1. Align the card with the card socket in the riser, and then slide it squarely into the socket in the riser.
  
  Make sure that the card is completely and squarely seated into the riser socket.
2. For Riser 2 or 3, close the side panel.
3. Swing the locking latch into place until it clicks into the locked position.
Install the riser into the controller module:
1. Align the lip of the riser with the underside of the controller module sheet metal.
2. Guide the riser along the pins in the controller module, and then lower the riser into the controller module.
3. Swing the locking latch down and click it into the locked position.
  
  When locked, the locking latch is flush with the top of the riser and the riser sits squarely in the controller module.
4. Reinsert any SFP modules that were removed from the PCIe cards.

Step 4: Reinstall the controller module

After you replace a component within the controller module, you must reinstall the controller module in the system chassis and boot it.

If you have not already done so, close the air duct:
1. Swing the air duct all the way down to the controller module.
2. Slide the air duct toward the risers until the locking tabs click into place.
3. Inspect the air duct to make sure that it is properly seated and locked into place.
  
  Locking tabs
  
  Slide plunger
Align the end of the controller module with the opening in the chassis, and then gently push the controller module halfway into the system.

Do not completely insert the controller module in the chassis until instructed to do so.
Cable the management and console ports only, so that you can access the system to perform the tasks in the following sections.

You will connect the rest of the cables to the controller module later in this procedure.
Complete the reinstallation of the controller module:
1. Firmly push the controller module into the chassis until it meets the midplane and is fully seated.
  
  The locking latches rise when the controller module is fully seated.
  
  Do not use excessive force when sliding the controller module into the chassis to avoid damaging the connectors.
2. Rotate the locking latches upward, tilting them so that they clear the locking pins, and then lower them into the locked position.
Plug the system cables and transceiver modules into the controller module and reinstall the cable management device.
Plug the power cables into the power supplies and reinstall the power cable retainers.

The controller module begins to boot as soon as it is connected to power. Be prepared to interrupt the boot process.

If your system has DC power supplies, make sure the thumbscrews on the power supply cable are tight.
Return the controller to normal operation by giving back its storage: storage failover giveback -ofnode impaired_node_name
If automatic giveback was disabled, reenable it: storage failover modify -node local -auto-giveback true

Step 5: Return the failed part to NetApp

Return the failed part to NetApp, as described in the RMA instructions shipped with the kit. See the Part Return and Replacements page for further information.