Powering off and powering on a data center

Contributors netapp-ivanad Download PDF of this page

You must know how to power off and power on a data center for the purpose of site maintenance or to relocate a site to another location.

If a site needs to be relocated and reconfigured (if you need to expand from a four-node to an eight-node cluster, for example), these tasks cannot be completed at the same time. This procedure only covers the steps that are required to perform site maintenance or to relocate a site without changing its configuration.

Note This procedure is for use in both MetroCluster IP and FC configurations.

Powering off a MetroCluster site

You must power off a site and all of the equipment before site maintenance or relocation can begin.

Steps
  1. Before you begin, check that any non-mirrored aggregates at the site are offline.

  2. Verify the operation of the MetroCluster configuration in ONTAP:

    1. Check whether the system is multipathed:
      node run -node node-name sysconfig -a

    2. Check for any health alerts on both clusters:
      system health alert show

    3. Confirm the MetroCluster configuration and that the operational mode is normal:
      metrocluster show

    4. Perform a MetroCluster check:
      metrocluster check run

    5. Display the results of the MetroCluster check:
      metrocluster check show

    6. Check for any health alerts on the switches (if present):
      storage switch show

    7. Run Config Advisor.

    8. After running Config Advisor, review the tool’s output and follow the recommendations in the output to address any issues discovered.

  3. Enter the following command to implement the switchover:

    metrocluster switchover

    The operation can take several minutes to complete.

    In MetroCluster FC configurations, the unmirrored aggregates will only be online after a switchover if the remote disks in the aggregate are accessible. If the ISLs fail, the local node may be unable to access the data in the unmirrored remote disks. The failure of an aggregate can lead to a reboot of the local node.

  4. Monitor and verify the completion of the switchover:

    metrocluster operation show

    cluster_A::*> metrocluster operation show
      Operation: Switchover
     Start time: 10/4/2012 19:04:13
    State: in-progress
       End time: -
         Errors:
    
    cluster_A::*> metrocluster operation show
      Operation: Switchover
     Start time: 10/4/2012 19:04:13
          State: successful
       End time: 10/4/2012 19:04:22
         Errors: -
  5. If you are in a MetroCluster IP configuration running ONTAP 9.6 or later, wait for the disaster site plexes to come online and the healing operations to automatically complete.

    In MetroCluster IP configurations running earlier versions of ONTAP, the disaster site nodes do not automatically boot to ONTAP and the plexes remain offline.

  6. Depending on your configuration and ONTAP version, identify and move offline affected plexes that are located at the disaster site.

    Note

    This step is required in the following configurations:.

    • All MetroCluster FC configurations.

    • MetroCluster IP configurations running ONTAP version 9.6 or later.

    You should move the following plexes offline:

    • Non-mirrored plexes residing on disks located at the disaster site.

      If you do not move the non-mirrored plexes at the disaster site offline, an outage might result when the disaster site is later powered off.

    • Mirrored plexes residing on disks located at the disaster site for aggregate mirroring. Once they are moved offline, the plexes are inaccessible.

      1. Identify the affected plexes.

        Plexes that are owned by nodes at the surviving site consist of Pool1 disks. Plexes that are owned by nodes at the disaster site consist of Pool0 disks.

        Note “Home” refers to the “home” owner, not the “current” owner.
        Cluster_A::> storage aggregate plex show -fields aggregate,status,is-online,Plex,pool
        aggregate    plex  status        is-online pool
        ------------ ----- ------------- --------- ----
        Node_B_1_aggr0 plex0 normal,active true     0
        Node_B_1_aggr0 plex1 normal,active true     1
        
        Node_B_2_aggr0 plex0 normal,active true     0
        Node_B_2_aggr0 plex5 normal,active true     1
        
        Node_B_1_aggr1 plex0 normal,active true     0
        Node_B_1_aggr1 plex3 normal,active true     1
        
        Node_B_2_aggr1 plex0 normal,active true     0
        Node_B_2_aggr1 plex1 normal,active true     1
        
        Node_A_1_aggr0 plex0 normal,active true     0
        Node_A_1_aggr0 plex4 normal,active true     1
        
        Node_A_1_aggr1 plex0 normal,active true     0
        Node_A_1_aggr1 plex1 normal,active true     1
        
        Node_A_2_aggr0 plex0 normal,active true     0
        Node_A_2_aggr0 plex4 normal,active true     1
        
        Node_A_2_aggr1 plex0 normal,active true     0
        Node_A_2_aggr1 plex1 normal,active true     1
        14 entries were displayed.
        
        Cluster_A::>

        The affected plexes are those that are remote to cluster A. The following table shows whether the disks are local or remote relative to cluster A:

        Node Disks in pool Should the disks be set offline? Example of plexes to be moved offline

        Node _A_1 and Node _A_2

        Disks in pool 0

        No. Disks are local to cluster A.

        -

        Disks in pool 1

        Yes. Disks are remote to cluster A.

        Node_A_1_aggr0/plex4

        Node_A_1_aggr1/plex1

        Node_A_2_aggr0/plex4

        Node_A_2_aggr1/plex1

        Node _B_1 and Node _B_2

        Disks in pool 0

        Yes. Disks are remote to cluster A.

        Node_B_1_aggr1/plex0

        Node_B_1_aggr0/plex0

        Node_B_2_aggr0/plex0

        Node_B_2_aggr1/plex0

        Disks in pool 1

        No. Disks are local to cluster A.

        -

      2. Move the affected plexes offline:
        storage aggregate plex offline

        storage aggregate plex offline -aggregate Node_B_1_aggr0 -plex plex0
        Note Perform this for all plexes that have disks that are remote to Cluster_A.
  7. Persistently offline the switchports according to the switch type.

    Note This step is only required for MetroCluster FC configurations. Skip this step if your configuration is a MetroCluster IP configuration or a stretched MetroCluster configuration with FC backend switches.
    Switch type Action

    If the FC switches are Brocade switches…​

    1. Use the portcfgpersistentdisable port command to persistently disable the ports as shown in the following example. This must be done on both switches at the surviving site.

       Switch_A_1:admin> portcfgpersistentdisable 14
       Switch_A_1:admin> portcfgpersistentdisable 15
       Switch_A_1:admin>
    2. Verify that the ports are disabled using the switchshow command shown in the following example:

       Switch_A_1:admin> switchshow
       switchName:	Switch_A_1
       switchType:	109.1
       switchState:	Online
       switchMode:	Native
       switchRole:	Principal
       switchDomain:	2
       switchId:	fffc02
       switchWwn:	10:00:00:05:33:88:9c:68
       zoning:		ON (T5_T6)
       switchBeacon:	OFF
       FC Router:	OFF
       FC Router BB Fabric ID:	128
       Address Mode:	0
      
        Index Port Address Media Speed State     Proto
        ==============================================
         ...
         14  14   020e00   id    16G   No_Light    FC  Disabled (Persistent)
         15  15   020f00   id    16G   No_Light    FC  Disabled (Persistent)
         ...
       Switch_A_1:admin>

    If the FC switches are Cisco switches…​

    1. Use the interface command to persistently disable the ports. The following example shows ports 14 and 15 being disabled:

       Switch_A_1# conf t
       Switch_A_1(config)# interface fc1/14-15
       Switch_A_1(config)# shut
      
       Switch_A_1(config-if)# end
       Switch_A_1# copy running-config startup-config
    2. Verify that the switch port is disabled using the show interface brief command as shown in the following example:

       Switch_A_1# show interface brief
       Switch_A_1
  8. Power off the site.

    The following equipment needs to be turned off in no specific order:

    Configuration type

    Equipment to be powered off

    In a MetroCluster IP configuration, power off…​

    • MetroCluster IP switches

    • Storage controllers

    • Storage shelves

    In a MetroCluster FC configuration, power off…​

    • MetroCluster FC switches

    • Storage controllers

    • Storage shelves

    • Atto FibreBridges (if present)

Relocating the powered-off site of the MetroCluster

Once the site is powered off, you can begin maintenance work. The procedure is the same whether the MetroCluster components are relocated within the same data center or relocated to a different data center.

  • The hardware should be cabled in the same way as the previous site.

  • If the Inter-Switch Link (ISL) speed, length, or number has changed, they all need to be reconfigured.

Steps
  1. Make sure that the cabling for all components is carefully recorded so that it can be correctly reconnected at the new location.

  2. Physically relocate all the hardware, storage controllers, FC and IP switches, FibreBridges, and storage shelves.

  3. Configure the ISL ports and verify the intersite connectivity.

    1. Power on the FC and IP switches.

      Note Do not power up any other equipment.
  4. Enable the ports.

    Note This step is only required in MetroCluster FC configurations. You can skip this step if your configuration is a MetroCluster IP configuration.

    Enable the ports according to the correct switch types in the following table:

    Switch type Command

    If the FC Switches are Brocade switches…​

    1. Use the portcfgpersistentenable port number command to persistently enable the port. This must be done on both switches at the surviving site.

      The following example shows ports 14 and 15 being enabled on Switch_A_1.

      switch_A_1:admin> portcfgpersistentenable 14
      switch_A_1:admin> portcfgpersistentenable 15
      switch_A_1:admin>
    2. Verify that the switch port is enabled: switchshow

      The following example shows that ports 14 and 15 are enabled:

      switch_A_1:admin> switchshow
      switchName:	Switch_A_1
      switchType:	109.1
      
      switchState:	Online
      switchMode:	Native
      switchRole:	Principal
      switchDomain:	2
      switchId:	fffc02
      switchWwn:	10:00:00:05:33:88:9c:68
      zoning:		ON (T5_T6)
      switchBeacon:	OFF
      FC Router:	OFF
      FC Router BB Fabric ID:	128
      Address Mode:	0
      
      Index Port Address Media Speed State     Proto
      ==============================================
       ...
       14  14   020e00   id    16G   Online      FC  E-Port  10:00:00:05:33:86:89:cb "Switch_A_1"
       15  15   020f00   id    16G   Online      FC  E-Port  10:00:00:05:33:86:89:cb "Switch_A_1" (downstream)
       ...
      switch_A_1:admin>

    If the FC Switches are Cisco switches…​

    1. Enter the interface command to enable the port.

      The following example shows ports 14 and 15 being enabled on Switch_A_1.

       switch_A_1# conf t
       switch_A_1(config)# interface fc1/14-15
       switch_A_1(config)# no shut
       switch_A_1(config-if)# end
       switch_A_1# copy running-config startup-config
    2. Verify that the switch port is enabled: show interface brief

       switch_A_1# show interface brief
       switch_A_1#
  5. Use tools on the switches (as they are available) to verify the intersite connectivity.

    Note You should only proceed if the links are properly configured and stable.
  6. Disable the links again if they are found to be stable.

    Disable the ports based on whether you are using Brocade or Cisco switches as shown in the following table:

    Switch type Command

    If the FC Switches are Brocade switches…​

    1. Enter the portcfgpersistentdisable port number command to persistently disable the port.

      This must be done on both switches at the surviving site. The following example shows ports 14 and 15 being disabled on Switch_A_1:

       switch_A_1:admin> portpersistentdisable 14
       switch_A_1:admin> portpersistentdisable 15
       switch_A_1:admin>
    2. Verify that the switch port is disabled: switchshow

      The following example shows that ports 14 and 15 are disabled:

      switch_A_1:admin> switchshow
      switchName:	Switch_A_1
      switchType:	109.1
      switchState:	Online
      switchMode:	Native
      switchRole:	Principal
      switchDomain:	2
      switchId:	fffc02
      switchWwn:	10:00:00:05:33:88:9c:68
      zoning:		ON (T5_T6)
      switchBeacon:	OFF
      FC Router:	OFF
      FC Router BB Fabric ID:	128
      Address Mode:	0
      
       Index Port Address Media Speed State     Proto
       ==============================================
        ...
        14  14   020e00   id    16G   No_Light    FC  Disabled (Persistent)
        15  15   020f00   id    16G   No_Light    FC  Disabled (Persistent)
        ...
      switch_A_1:admin>

    If the FC Switches are Cisco switches…​

    1. Disable the port using the interface command.

      The following example shows ports fc1/14 and fc1/15 being disabled on Switch A_1:

      switch_A_1# conf t
      
      switch_A_1(config)# interface fc1/14-15
      switch_A_1(config)# shut
      switch_A_1(config-if)# end
      switch_A_1# copy running-config startup-config
    2. Verify that the switch port is disabled using the show interface brief command.

        switch_A_1# show interface brief
        switch_A_1#

Powering on the MetroCluster configuration and returning to normal operation

After maintenance has been completed or the site has been moved, you must power on the site and reestablish the MetroCluster configuration.

Steps
  1. Power on the switches.

    Switches should be powered on first. They might have been powered on during the previous step if the site was relocated.

    1. Reconfigure the Inter-Switch Link (ISL) if required or if this was not completed as part of the relocation.

    2. Enable the ISL if fencing was completed.

    3. Verify the ISL.

  2. Power on the shelves and allow enough time for them to power on completely.

  3. Power on the FibreBridge bridges.

    Note You can skip this step if your configuration is a MetroCluster IP configuration.
    1. On the FC switches, verify that the ports connecting the bridges are coming online.

      You can use a command such as switchshow for Brocade switches, and show interface brief for Cisco switches.

    2. Verify that the shelves and disks on the bridges are clearly visible.

      You can use a command such as sastargets on the ATTO command-line interface (CLI).

  4. Enable the ISLs on the FC switches.

    Note Skip this step if your configuration is a MetroCluster IP configuration.

    Enable the ports based on whether you are using Brocade or Cisco switches as shown in the following table:

    Switch type Command

    If the FC Switches are Brocade switches…​

    1. Enter the portcfgpersistentenable port command to persistently enable the ports. This must be done on both switches at the surviving site.

      The following example shows ports 14 and 15 being enabled on Switch_A_1:

       Switch_A_1:admin> portcfgpersistentenable 14
       Switch_A_1:admin> portcfgpersistentenable 15
       Switch_A_1:admin>
    2. Verify that the switch port is enabled using the
      switchshow command:

      switch_A_1:admin> switchshow
       switchName:	Switch_A_1
       switchType:	109.1
       switchState:	Online
       switchMode:	Native
       switchRole:	Principal
       switchDomain:	2
       switchId:	fffc02
       switchWwn:	10:00:00:05:33:88:9c:68
       zoning:		ON (T5_T6)
       switchBeacon:	OFF
       FC Router:	OFF
       FC Router BB Fabric ID:	128
       Address Mode:	0
      
        Index Port Address Media Speed State     Proto
        ==============================================
         ...
         14  14   020e00   id    16G   Online      FC  E-Port  10:00:00:05:33:86:89:cb "Switch_A_1"
         15  15   020f00   id    16G   Online      FC  E-Port  10:00:00:05:33:86:89:cb "Switch_A_1" (downstream)
         ...
       switch_A_1:admin>

    If the FC Switches are Cisco switches…​

    1. Use the interface command to enable the ports.

      The following example shows port fc1/14 and fc1/15 being enabled on Switch A_1:

       switch_A_1# conf t
       switch_A_1(config)# interface fc1/14-15
       switch_A_1(config)# no shut
       switch_A_1(config-if)# end
       switch_A_1# copy running-config startup-config
    2. Verify that the switch port is disabled:

      switch_A_1# show interface brief
      switch_A_1#
  5. Verify that the storage is now visible.

    Select the appropriate method of determining whether the storage is visible based on whether you have a MetroCluster IP or FC configuration:

    Configuration Step

    If your configuration is a MetroCluster IP…​

    Verify that the local storage is visible from the node Maintenance mode.

    If your configuration is a MetroCluster FC…​

    Verify that the storage is visible from the surviving site. Put the offline plexes back online. This restarts the resync operations and reestablishes the SyncMirror.

  6. Reestablish the MetroCluster configuration.

    Follow the instructions in the MetroCluster Disaster and Recovery Guide to perform healing and switchback operations according to your MetroCluster configuration.