ONTAP MetroCluster

Powering off and powering on a data center

Contributors netapp-folivia netapp-thomi netapp-ahibbard netapp-martyh
PDFs

You must know how to power off and power on a data center for the purpose of site maintenance or to relocate a site to another location.

If a site needs to be relocated and reconfigured (if you need to expand from a four-node to an eight-node cluster, for example), these tasks cannot be completed at the same time. This procedure only covers the steps that are required to perform site maintenance or to relocate a site without changing its configuration.

The following diagram shows a MetroCluster configuration. Cluster_B is powered off for maintenance.

Note This procedure is for use in both MetroCluster IP and FC configurations.
power on off data center

Powering off a MetroCluster site

You must power off a site and all of the equipment before site maintenance or relocation can begin.

About this task

All the commands in the following steps are issued from the site to remain powered on.

Steps

  1. Before you begin, check that any non-mirrored aggregates at the site are offline.

  2. Verify the operation of the MetroCluster configuration in ONTAP:

    1. Check whether the system is multipathed:

      node run -node node-name sysconfig -a

    2. Check for any health alerts on both clusters:

      system health alert show

    3. Confirm the MetroCluster configuration and that the operational mode is normal:

      metrocluster show

    4. Perform a MetroCluster check:
      metrocluster check run

    5. Display the results of the MetroCluster check:

      metrocluster check show

    6. Check for any health alerts on the switches (if present):

      storage switch show

    7. Run Config Advisor.

      NetApp Downloads: Config Advisor

    8. After running Config Advisor, review the tool’s output and follow the recommendations in the output to address any issues discovered.

  3. From the site you want to remain up implement the switchover:

    metrocluster switchover

    cluster_A::*> metrocluster switchover

    The operation can take several minutes to complete.

    In MetroCluster FC configurations, the unmirrored aggregates will only be online after a switchover if the remote disks in the aggregate are accessible. If the ISLs fail, the local node may be unable to access the data in the unmirrored remote disks. The failure of an aggregate can lead to a reboot of the local node.

  4. Monitor and verify the completion of the switchover:

    metrocluster operation show

    cluster_A::*> metrocluster operation show
  Operation: Switchover
 Start time: 10/4/2012 19:04:13
State: in-progress
   End time: -
     Errors:

cluster_A::*> metrocluster operation show
  Operation: Switchover
 Start time: 10/4/2012 19:04:13
      State: successful
   End time: 10/4/2012 19:04:22
     Errors: -

  5. If you have a MetroCluster IP configuration running ONTAP 9.6 or later, wait for the disaster site plexes to come online and the healing operations to automatically complete.

    In MetroCluster IP configurations running earlier versions of ONTAP, the disaster site nodes do not automatically boot to ONTAP and the plexes remain offline. For MetroCluster FC configurations, all plexes remain online and you do not need to online the plexes.

  6. Move any volumes and LUNs that belong to unmirrored aggregates offline.

    1. Move the volumes offline.

      cluster_A::* volume offline <volume name>

    2. Move the LUNs offline.

      cluster_A::* lun offline lun_path <lun_path>

  7. Move unmirrored aggregates offline: storage aggregate offline

    cluster_A*::> storage aggregate offline -aggregate <aggregate-name>

  8. Depending on your configuration and ONTAP version, identify and move offline affected plexes that are located at the disaster site (Cluster_B).

    Note

    This step is required in the following configurations:.

    • All MetroCluster FC configurations.

    • MetroCluster IP configurations running ONTAP version 9.6 or later.

    You should move the following plexes offline:

    • Non-mirrored plexes residing on disks located at the disaster site.

      If you do not move the non-mirrored plexes at the disaster site offline, an outage might result when the disaster site is later powered off.

    • Mirrored plexes residing on disks located at the disaster site for aggregate mirroring. After they are moved offline, the plexes are inaccessible.

    1. Identify the affected plexes.

      Plexes that are owned by nodes at the surviving site consist of Pool1 disks. Plexes that are owned by nodes at the disaster site consist of Pool0 disks.

      Cluster_A::> storage aggregate plex show -fields aggregate,status,is-online,Plex,pool
aggregate    plex  status        is-online pool
------------ ----- ------------- --------- ----
Node_B_1_aggr0 plex0 normal,active true     0
Node_B_1_aggr0 plex1 normal,active true     1

Node_B_2_aggr0 plex0 normal,active true     0
Node_B_2_aggr0 plex5 normal,active true     1

Node_B_1_aggr1 plex0 normal,active true     0
Node_B_1_aggr1 plex3 normal,active true     1

Node_B_2_aggr1 plex0 normal,active true     0
Node_B_2_aggr1 plex1 normal,active true     1

Node_A_1_aggr0 plex0 normal,active true     0
Node_A_1_aggr0 plex4 normal,active true     1

Node_A_1_aggr1 plex0 normal,active true     0
Node_A_1_aggr1 plex1 normal,active true     1

Node_A_2_aggr0 plex0 normal,active true     0
Node_A_2_aggr0 plex4 normal,active true     1

Node_A_2_aggr1 plex0 normal,active true     0
Node_A_2_aggr1 plex1 normal,active true     1
14 entries were displayed.

Cluster_A::>

      The affected plexes are those that are remote to cluster A. The following table shows whether the disks are local or remote relative to cluster A:

      Node

      Disks in pool

      Should the disks be set offline?

      Example of plexes to be moved offline

      Node _A_1 and Node _A_2

      Disks in pool 0

      No. Disks are local to cluster A.

      -

      Disks in pool 1

      Yes. Disks are remote to cluster A.

      Node_A_1_aggr0/plex4

      Node_A_1_aggr1/plex1

      Node_A_2_aggr0/plex4

      Node_A_2_aggr1/plex1

      Node _B_1 and Node _B_2

      Disks in pool 0

      Yes. Disks are remote to cluster A.

      Node_B_1_aggr1/plex0

      Node_B_1_aggr0/plex0

      Node_B_2_aggr0/plex0

      Node_B_2_aggr1/plex0

      Disks in pool 1

      No. Disks are local to cluster A.

      -

    2. Move the affected plexes offline:

      storage aggregate plex offline

      storage aggregate plex offline -aggregate Node_B_1_aggr0 -plex plex0
      Note Perform this for all plexes that have disks that are remote to Cluster_A.

  9. Persistently offline the switchports according to the switch type.

    Note This step is only required for MetroCluster FC configurations. Skip this step if your configuration is a MetroCluster IP configuration or a stretched MetroCluster configuration with FC backend switches.

    Switch type

    Action

    If the FC switches are Brocade switches…​

    1. Use the portcfgpersistentdisable port command to persistently disable the ports as shown in the following example. This must be done on both switches at the surviving site.

       Switch_A_1:admin> portcfgpersistentdisable 14
 Switch_A_1:admin> portcfgpersistentdisable 15
 Switch_A_1:admin>

    2. Verify that the ports are disabled using the switchshow command shown in the following example:

       Switch_A_1:admin> switchshow
 switchName:	Switch_A_1
 switchType:	109.1
 switchState:	Online
 switchMode:	Native
 switchRole:	Principal
 switchDomain:	2
 switchId:	fffc02
 switchWwn:	10:00:00:05:33:88:9c:68
 zoning:		ON (T5_T6)
 switchBeacon:	OFF
 FC Router:	OFF
 FC Router BB Fabric ID:	128
 Address Mode:	0

  Index Port Address Media Speed State     Proto
  ==============================================
   ...
   14  14   020e00   id    16G   No_Light    FC  Disabled (Persistent)
   15  15   020f00   id    16G   No_Light    FC  Disabled (Persistent)
   ...
 Switch_A_1:admin>

    If the FC switches are Cisco switches…​

    1. Use the interface command to persistently disable the ports. The following example shows ports 14 and 15 being disabled:

       Switch_A_1# conf t
 Switch_A_1(config)# interface fc1/14-15
 Switch_A_1(config)# shut

 Switch_A_1(config-if)# end
 Switch_A_1# copy running-config startup-config

    2. Verify that the switch port is disabled using the show interface brief command as shown in the following example:

       Switch_A_1# show interface brief
 Switch_A_1

  10. Power off the site.

    The following equipment needs to be turned off in no specific order:

    Configuration type

    Equipment to be powered off

    In a MetroCluster IP configuration, power off…​

    • MetroCluster IP switches

    • Storage controllers

    • Storage shelves

    In a MetroCluster FC configuration, power off…​

    • MetroCluster FC switches

    • Storage controllers

    • Storage shelves

    • Atto FibreBridges (if present)

Relocating the powered-off site of the MetroCluster

After the site is powered off, you can begin maintenance work. The procedure is the same whether the MetroCluster components are relocated within the same data center or relocated to a different data center.

  • The hardware should be cabled in the same way as the previous site.

  • If the Inter-Switch Link (ISL) speed, length, or number has changed, they all need to be reconfigured.

Steps

  1. Make sure that the cabling for all components is carefully recorded so that it can be correctly reconnected at the new location.

  2. Physically relocate all the hardware, storage controllers, FC and IP switches, FibreBridges, and storage shelves.

  3. Configure the ISL ports and verify the intersite connectivity.

    1. Power on the FC and IP switches.

      Note Do not power up any other equipment.

    2. Enable the ports.

      Note This step is only required in MetroCluster FC configurations. You can skip this step if your configuration is a MetroCluster IP configuration.

      Enable the ports according to the correct switch types in the following table:

      Switch type

      Command

      If the FC Switches are Brocade switches…​

      1. Use the portcfgpersistentenable port number command to persistently enable the port. This must be done on both switches at the surviving site.

        The following example shows ports 14 and 15 being enabled on Switch_A_1.

        switch_A_1:admin> portcfgpersistentenable 14
switch_A_1:admin> portcfgpersistentenable 15
switch_A_1:admin>

      2. Verify that the switch port is enabled: switchshow

        The following example shows that ports 14 and 15 are enabled:

        switch_A_1:admin> switchshow
switchName:	Switch_A_1
switchType:	109.1

switchState:	Online
switchMode:	Native
switchRole:	Principal
switchDomain:	2
switchId:	fffc02
switchWwn:	10:00:00:05:33:88:9c:68
zoning:		ON (T5_T6)
switchBeacon:	OFF
FC Router:	OFF
FC Router BB Fabric ID:	128
Address Mode:	0

Index Port Address Media Speed State     Proto
==============================================
 ...
 14  14   020e00   id    16G   Online      FC  E-Port  10:00:00:05:33:86:89:cb "Switch_A_1"
 15  15   020f00   id    16G   Online      FC  E-Port  10:00:00:05:33:86:89:cb "Switch_A_1" (downstream)
 ...
switch_A_1:admin>

      If the FC Switches are Cisco switches…​

      1. Enter the interface command to enable the port.

        The following example shows ports 14 and 15 being enabled on Switch_A_1.

         switch_A_1# conf t
 switch_A_1(config)# interface fc1/14-15
 switch_A_1(config)# no shut
 switch_A_1(config-if)# end
 switch_A_1# copy running-config startup-config

      2. Verify that the switch port is enabled: show interface brief

         switch_A_1# show interface brief
 switch_A_1#

  4. Use tools on the switches (as they are available) to verify the intersite connectivity.

    Note You should only proceed if the links are properly configured and stable.

  5. Disable the links again if they are found to be stable.

    Disable the ports based on whether you are using Brocade or Cisco switches as shown in the following table:

    Switch type

    Command

    If the FC Switches are Brocade switches…​

    1. Enter the portcfgpersistentdisable port number command to persistently disable the port.

      This must be done on both switches at the surviving site. The following example shows ports 14 and 15 being disabled on Switch_A_1:

       switch_A_1:admin> portpersistentdisable 14
 switch_A_1:admin> portpersistentdisable 15
 switch_A_1:admin>

    2. Verify that the switch port is disabled: switchshow

      The following example shows that ports 14 and 15 are disabled:

      switch_A_1:admin> switchshow
switchName:	Switch_A_1
switchType:	109.1
switchState:	Online
switchMode:	Native
switchRole:	Principal
switchDomain:	2
switchId:	fffc02
switchWwn:	10:00:00:05:33:88:9c:68
zoning:		ON (T5_T6)
switchBeacon:	OFF
FC Router:	OFF
FC Router BB Fabric ID:	128
Address Mode:	0

 Index Port Address Media Speed State     Proto
 ==============================================
  ...
  14  14   020e00   id    16G   No_Light    FC  Disabled (Persistent)
  15  15   020f00   id    16G   No_Light    FC  Disabled (Persistent)
  ...
switch_A_1:admin>

    If the FC Switches are Cisco switches…​

    1. Disable the port using the interface command.

      The following example shows ports fc1/14 and fc1/15 being disabled on Switch A_1:

      switch_A_1# conf t

switch_A_1(config)# interface fc1/14-15
switch_A_1(config)# shut
switch_A_1(config-if)# end
switch_A_1# copy running-config startup-config

    2. Verify that the switch port is disabled using the show interface brief command.

        switch_A_1# show interface brief
  switch_A_1#

Powering on the MetroCluster configuration and returning to normal operation

After maintenance has been completed or the site has been moved, you must power on the site and reestablish the MetroCluster configuration.

About this task

All the commands in the following steps are issued from the site that you power on.

Steps

  1. Power on the switches.

    Switches should be powered on first. They might have been powered on during the previous step if the site was relocated.

    1. Reconfigure the Inter-Switch Link (ISL) if required or if this was not completed as part of the relocation.

    2. Enable the ISL if fencing was completed.

    3. Verify the ISL.

  2. Power on the storage controllers.

  3. Power on the shelves and allow enough time for them to power on completely.

  4. Power on the FibreBridge bridges.

    Note You can skip this step if your configuration is a MetroCluster IP configuration.

    1. On the FC switches, verify that the ports connecting the bridges are coming online.

      You can use a command such as switchshow for Brocade switches, and show interface brief for Cisco switches.

    2. Verify that the shelves and disks on the bridges are clearly visible.

      You can use a command such as sastargets on the ATTO command-line interface (CLI).

  5. Enable the ISLs on the FC switches.

    Note Skip this step if your configuration is a MetroCluster IP configuration.

    Enable the ports based on whether you are using Brocade or Cisco switches as shown in the following table:

    Switch type

    Command

    If the FC Switches are Brocade switches…​

    1. Enter the portcfgpersistentenable port command to persistently enable the ports. This must be done on both switches at the surviving site.

      The following example shows ports 14 and 15 being enabled on Switch_A_1:

       Switch_A_1:admin> portcfgpersistentenable 14
 Switch_A_1:admin> portcfgpersistentenable 15
 Switch_A_1:admin>

    2. Verify that the switch port is enabled using the
      switchshow command:

      switch_A_1:admin> switchshow
 switchName:	Switch_A_1
 switchType:	109.1
 switchState:	Online
 switchMode:	Native
 switchRole:	Principal
 switchDomain:	2
 switchId:	fffc02
 switchWwn:	10:00:00:05:33:88:9c:68
 zoning:		ON (T5_T6)
 switchBeacon:	OFF
 FC Router:	OFF
 FC Router BB Fabric ID:	128
 Address Mode:	0

  Index Port Address Media Speed State     Proto
  ==============================================
   ...
   14  14   020e00   id    16G   Online      FC  E-Port  10:00:00:05:33:86:89:cb "Switch_A_1"
   15  15   020f00   id    16G   Online      FC  E-Port  10:00:00:05:33:86:89:cb "Switch_A_1" (downstream)
   ...
 switch_A_1:admin>

    If the FC Switches are Cisco switches…​

    1. Use the interface command to enable the ports.

      The following example shows port fc1/14 and fc1/15 being enabled on Switch A_1:

       switch_A_1# conf t
 switch_A_1(config)# interface fc1/14-15
 switch_A_1(config)# no shut
 switch_A_1(config-if)# end
 switch_A_1# copy running-config startup-config

    2. Verify that the switch port is disabled:

      switch_A_1# show interface brief
switch_A_1#

  6. Verify that the storage is now visible.

    Select the appropriate method of determining whether the storage is visible based on whether you have a MetroCluster IP or FC configuration:

    If your configuration is…​

    Then perform this step…​

    MetroCluster IP configuration

    Verify that the local storage is visible from the node Maintenance mode.

    MetroCluster FC configuration

    Verify that the storage is visible from the surviving site. Put the offline plexes back online. This restarts the resync operations and reestablishes the SyncMirror.

  7. Reestablish the MetroCluster configuration.

    Follow the instructions in MetroCluster management and disaster recovery to perform healing and switchback operations according to your MetroCluster configuration.