Recable the system and reassign disks - FAS8300 and FAS8700

Contributors netapp-martyh thrisun dougthomp

You must complete a series of tasks before restoring your system to full operation.

Step 1: Recable the system

After running diagnostics, you must recable the controller module’s storage and network connections.

Steps
  1. Recable the system.

  2. Verify that the cabling is correct by using Active IQ Config Advisor.

    1. Download and install Config Advisor.

    2. Enter the information for the target system, and then click Collect Data.

    3. Click the Cabling tab, and then examine the output. Make sure that all disk shelves are displayed and all disks appear in the output, correcting any cabling issues you find.

    4. Check other cabling by clicking the appropriate tab, and then examining the output from Config Advisor.

Step 2: Reassign disks

If the storage system is in an HA pair, the system ID of the new controller module is automatically assigned to the disks when the giveback occurs at the end of the procedure. In a stand-alone system, you must manually reassign the ID to the disks.

You must use the correct procedure for your configuration:

Controller redundancy Then use this procedure…​

HA pair

Two-node MetroCluster configuration

Option 1: Verify the system ID change on an HA system

You must confirm the system ID change when you boot the replacement node and then verify that the change was implemented.

This procedure applies only to systems running ONTAP in an HA pair.

  1. If the replacement node is in Maintenance mode (showing the *> prompt, exit Maintenance mode and go to the LOADER prompt: halt

  2. From the LOADER prompt on the replacement node, boot the node, entering y if you are prompted to override the system ID due to a system ID mismatch:

  3. Wait until the Waiting for giveback…​ message is displayed on the replacement node console and then, from the healthy node, verify that the new partner system ID has been automatically assigned: storage failover show

    In the command output, you should see a message that the system ID has changed on the impaired node, showing the correct old and new IDs. In the following example, node2 has undergone replacement and has a new system ID of 151759706.

    node1> storage failover show
                                        Takeover
    Node              Partner           Possible     State Description
    ------------      ------------      --------     -------------------------------------
    node1             node2             false        System ID changed on partner (Old:
                                                      151759755, New: 151759706), In takeover
    node2             node1             -            Waiting for giveback (HA mailboxes)
  4. From the healthy node, verify that any coredumps are saved:

    1. Change to the advanced privilege level: set -privilege advanced

      You can respond Y when prompted to continue into advanced mode. The advanced mode prompt appears (*>).

    2. Save any coredumps: system node run -node local-node-name partner savecore

    3. Wait for savecore command to complete before issuing the giveback.

      You can enter the following command to monitor the progress of the savecore command: system node run -node local-node-name partner savecore -s

    4. Return to the admin privilege level: set -privilege admin

  5. Give back the node:

    1. From the healthy node, give back the replaced node’s storage: storage failover giveback -ofnode replacement_node_name

      the replacement node takes back its storage and completes booting.

      If you are prompted to override the system ID due to a system ID mismatch, you should enter y.

      Note If the giveback is vetoed, you can consider overriding the vetoes.
    2. After the giveback has been completed, confirm that the HA pair is healthy and that takeover is possible: storage failover show

      The output from the storage failover show command. should not include the System ID changed on partner message.

  6. Verify that the disks were assigned correctly: storage disk show -ownership

    The disks belonging to the replacement node should show the new system ID. In the following example, the disks owned by node1 now show the new system ID, 1873775277:

    node1> storage disk show -ownership
    
    Disk  Aggregate Home  Owner  DR Home  Home ID    Owner ID  DR Home ID Reserver  Pool
    ----- ------    ----- ------ -------- -------    -------    -------  ---------  ---
    1.0.0  aggr0_1  node1 node1  -        1873775277 1873775277  -       1873775277 Pool0
    1.0.1  aggr0_1  node1 node1           1873775277 1873775277  -       1873775277 Pool0
    .
    .
    .

Option 2: Manually reassign the system ID on systems in a two-node MetroCluster configuration

In a two-node MetroCluster configuration running ONTAP, you must manually reassign disks to the new controller’s system ID before you return the system to normal operating condition.

About this task

This procedure applies only to systems in a two-node MetroCluster configuration running ONTAP.

You must be sure to issue the commands in this procedure on the correct node:

  • The impaired node is the node on which you are performing maintenance.

  • The replacement node is the new node that replaced the impaired node as part of this procedure.

  • The healthy node is the DR partner of the impaired node.

Steps
  1. If you have not already done so, reboot the replacement node, interrupt the boot process by entering Ctrl-C, and then select the option to boot to Maintenance mode from the displayed menu.

    You must enter Y when prompted to override the system ID due to a system ID mismatch.

  2. View the old system IDs from the healthy node: metrocluster node show -fields node-systemid,dr-partner-systemid

    In this example, the Node_B_1 is the old node, with the old system ID of 118073209:

    dr-group-id cluster         node                 node-systemid dr-partner-systemid
     ----------- --------------------- -------------------- ------------- -------------------
     1           Cluster_A             Node_A_1             536872914     118073209
     1           Cluster_B             Node_B_1             118073209     536872914
     2 entries were displayed.
  3. View the new system ID at the Maintenance mode prompt on the impaired node: disk show

    In this example, the new system ID is 118065481:

    Local System ID: 118065481
        ...
        ...
  4. Reassign disk ownership (for FAS systems) or LUN ownership (for FlexArray systems), by using the system ID information obtained from the disk show command: disk reassign -s old system ID

    In the case of the preceding example, the command is: disk reassign -s 118073209

    You can respond Y when prompted to continue.

  5. Verify that the disks (or FlexArray LUNs) were assigned correctly: disk show -a

    Verify that the disks belonging to the replacement node show the new system ID for the replacement node. In the following example, the disks owned by system-1 now show the new system ID, 118065481:

    *> disk show -a
    Local System ID: 118065481
    
      DISK     OWNER                 POOL   SERIAL NUMBER  HOME
    -------    -------------         -----  -------------  -------------
    disk_name   system-1  (118065481) Pool0  J8Y0TDZC       system-1  (118065481)
    disk_name   system-1  (118065481) Pool0  J8Y09DXC       system-1  (118065481)
    .
    .
    .
  6. From the healthy node, verify that any coredumps are saved:

    1. Change to the advanced privilege level: set -privilege advanced

      You can respond Y when prompted to continue into advanced mode. The advanced mode prompt appears (*>).

    2. Verify that the coredumps are saved: system node run -node local-node-name partner savecore

      If the command output indicates that savecore is in progress, wait for savecore to complete before issuing the giveback. You can monitor the progress of the savecore using the system node run -node local-node-name partner savecore -s command.</info>.

    3. Return to the admin privilege level: set -privilege admin

  7. If the replacement node is in Maintenance mode (showing the *> prompt), exit Maintenance mode and go to the LOADER prompt: halt

  8. Boot the replacement node: boot_ontap

  9. After the replacement node has fully booted, perform a switchback: metrocluster switchback

  10. Verify the MetroCluster configuration: metrocluster node show - fields configuration-state

    node1_siteA::> metrocluster node show -fields configuration-state
    
    dr-group-id            cluster node           configuration-state
    -----------            ---------------------- -------------- -------------------
    1 node1_siteA          node1mcc-001           configured
    1 node1_siteA          node1mcc-002           configured
    1 node1_siteB          node1mcc-003           configured
    1 node1_siteB          node1mcc-004           configured
    
    4 entries were displayed.
  11. Verify the operation of the MetroCluster configuration in Data ONTAP:

    1. Check for any health alerts on both clusters: system health alert show

    2. Confirm that the MetroCluster is configured and in normal mode: metrocluster show

    3. Perform a MetroCluster check: metrocluster check run

    4. Display the results of the MetroCluster check: metrocluster check show

    5. Run Config Advisor. Go to the Config Advisor page on the NetApp Support Site at support.netapp.com/NOW/download/tools/config_advisor/.

      After running Config Advisor, review the tool’s output and follow the recommendations in the output to address any issues discovered.

  12. Simulate a switchover operation:

    1. From any node’s prompt, change to the advanced privilege level: set -privilege advanced

      You need to respond with y when prompted to continue into advanced mode and see the advanced mode prompt (*>).

    2. Perform the switchback operation with the -simulate parameter: metrocluster switchover -simulate

    3. Return to the admin privilege level: set -privilege admin