Skip to main content

Install and boot node4

Contributors netapp-pcarriga
PDFs

You install node4 in the rack, transfer node2's connections to node4, boot node4, and install ONTAP. You then reassign any of node2’s spare disks, any disks belonging to the root volume, and any non-root aggregates that were not relocated to node3 earlier in the process, as outlined in this section.

About this task

The relocation operation is paused at the beginning of this stage. This process is mostly automated; the operation pauses to enable you to check its status. You must manually resume the operation.

You need to netboot node4 if it does not have the same version of ONTAP 9 that is installed on node2. After you install node4, boot it from the ONTAP 9 image stored on the web server. You can then download the correct files to the boot media device for subsequent system boots, by following the instructions in Prepare for netboot.

Steps

  1. Make sure that node4 has sufficient rack space.

    If node4 is in a separate chassis from node2, you can put node4 in the same location as node3. If node2 and node4 are in the same chassis, then node4 is already in its appropriate rack location.

  2. Install node4 in the rack, following the instructions in the Installation and Setup Instructions for the node model.

  3. Cable node4, moving the connections from node2 to node4.

    Cable the following connections, using the instructions in the Installation and Setup Instructions or the FlexArray Virtualization Installation Requirements and Reference for the node4 platform, the appropriate disk shelf document, and High Availability management.

    Refer to References to link to the FlexArray Virtualization Installation Requirements and Reference and High Availability management.

    • Console (remote management port)

    • Cluster and HA ports

    • Data ports

    • Cluster and node management ports

    • Serial-attached SCSI (SAS) and Ethernet storage ports

    • SAN configurations: iSCSI Ethernet, FC, and NVMe/FC switch ports

      Note

      You might need to change the interconnect cables between the old and new controllers to allow interoperability between the different controller and card models. Refer to the system installation procedures for a cabling map of the Ethernet storage shelves for your systems.

      For controllers introduced in ONTAP 9.15.1 and later, cluster and HA interconnects use the same ports. For switch connected configurations, it is required to connect similar ports to the same cluster switches. For example, when upgrading to an AFF A1K from an existing controller, you should connect e1a ports on both nodes to one switch and e7a ports on both nodes to the second switch.

  4. Turn on the power to node4, and then interrupt the boot process by pressing Ctrl-C at the console terminal to access the boot environment prompt.

    Note When you boot node4, you might see the following warning message: 
    WARNING: The battery is unfit to retain data during a power outage. This is likely
         because the battery is discharged but could be due to other temporary
         conditions.
         When the battery is ready, the boot process will complete
         and services will be engaged. To override this delay, press 'c' followed
         by 'Enter'

  5. If you see the warning message in Step 4, take the following actions:

    1. Check for any console messages that might indicate a problem other than a low NVRAM battery, and, if necessary, take any required corrective action.

    2. Allow the battery to charge and the boot process to complete.

      Warning Attention: Do not override the delay; failure to allow the battery to charge could result in a loss of data.
      Note Refer to Prepare for netboot.

  6. Configure the netboot connection by choosing one of the following actions.

    Note You must use the management port and IP as the netboot connection. Do not use a data LIF IP or a data outage might occur while the upgrade is being performed.
    If Dynamic Host Configuration Protocol (DHCP) is…​ Then…​

    Running

    Configure the connection automatically by using the following command at the boot environment prompt:

    ifconfig e0M -auto

    Not running

    Manually configure the connection by entering the following command at the boot environment prompt:
    ifconfig e0M -addr=filer_addr -mask=netmask -gw=gateway -dns=dns_addr -domain=dns_domain

    filer_addr is the IP address of the storage system (mandatory).
    netmask is the network mask of the storage system (mandatory).
    gateway is the gateway for the storage system (mandatory).
    dns_addr is the IP address of a name server on your network (optional).
    dns_domain is the DNS domain name (optional).

    Note Other parameters might be necessary for your interface. Enter help ifconfig at the firmware prompt for details.

  7. Perform netboot on node4:

    netboot http://<web_server_ip/path_to_web-accessible_directory>/<ontap_version>_image.tgz

    The <path_to_the_web-accessible_directory> should lead to where you downloaded the <ontap_version>_image.tgz in Step 1 in the section Prepare for netboot.

    Note Do not interrupt the boot.

  8. From the boot menu, select option (7) Install new software first.

    This menu option downloads and installs the new ONTAP image to the boot device.

    Disregard the following message:

    This procedure is not supported for Non-Disruptive Upgrade on an HA pair

    The note applies to nondisruptive upgrades of ONTAP, and not upgrades of controllers.

    Note Always use netboot to update the new node to the desired image. If you use another method to install the image on the new controller, the incorrect image might install. This issue applies to all ONTAP releases. The netboot procedure combined with option (7) Install new software wipes the boot media and places the same ONTAP version on both image partitions.

  9. If you are prompted to continue the procedure, enter y, and when prompted for the package, enter the URL:

    http://<web_server_ip/path_to_web-accessible_directory>/<ontap_version>_image.tgz

  10. Complete the following substeps to reboot the controller module:

    1. Enter n to skip the backup recovery when you see the following prompt:

      Do you want to restore the backup configuration now? {y|n}

    2. Reboot by entering y when you see the following prompt:

      The node must be rebooted to start using the newly installed software. Do you want to reboot now? {y|n}

      The controller module reboots but stops at the boot menu because the boot device was reformatted, and the configuration data must be restored.

  11. Select maintenance mode 5 from the boot menu and enter y when you are prompted to continue with the boot.

  12. Verify that the controller and chassis are configured as HA:

    ha-config show

    The following example shows the output of the ha-config show command:

    Chassis HA configuration: ha
Controller HA configuration: ha
    Note System records in a PROM whether they are in an HA pair or stand-alone configuration. The state must be the same on all components within the stand-alone system or HA pair.

  13. If the controller and chassis are not configured as HA, use the following commands to correct the configuration:

    ha-config modify controller ha

    ha-config modify chassis ha

  14. Confirm that all Ethernet ports used to connect to the Ethernet shelves are configured as storage:

    storage port show

    The output displayed depends on the system configuration. The following output example is for a node with a single storage card in slot11. The output for your system might be different:

    *> storage port show
Port Type Mode    Speed(Gb/s) State    Status  VLAN ID
---- ---- ------- ----------- -------- ------- -------
e11a ENET storage 100 Gb/s    enabled  online  30
e11b ENET storage 100 Gb/s    enabled  online  30

  15. Modify the ports that are not set to storage:

    storage port modify -p <port> -m storage

    All Ethernet ports connected to storage shelves must be configured as storage to allow access to the disks and shelves.

  16. Exit maintenance mode:

    halt

    Interrupt the autoboot by pressing Ctrl-C at the boot environment prompt.

  17. On node3, check the system date, time, and time zone:

    date

  18. On node4, check the date by using the following command at the boot environment prompt:

    show date

  19. If necessary, set the date on node4:

    set date <mm/dd/yyyy>

  20. On node4, check the time by using the following command at the boot environment prompt:

    show time

  21. If necessary, set the time on node4:

    set time <hh:mm:ss>

  22. In boot loader, set the partner system ID on node4:

    setenv partner-sysid <node3_sysid>

    For node4, partner-sysid must be that of node3.

    Save the settings:

    saveenv

  23. Verify the partner-sysid for node4:

    printenv partner-sysid

  24. If you have NetApp Storage Encryption (NSE) drives installed, perform the following steps.

    Note If you have not already done so earlier in the procedure, see the Knowledge Base article How to tell if a drive is FIPS certified to determine the type of self-encrypting drives that are in use.

    1. Set bootarg.storageencryption.support to true or false.

      If the following drives are in use… Then…

      NSE drives that conform to FIPS 140-2 Level 2 self-encryption requirements

      setenv bootarg.storageencryption.support true

      NetApp non-FIPS SEDs

      setenv bootarg.storageencryption.support false

    2. Go to the special boot menu and select option (10) Set Onboard Key Manager recovery secrets.

      Enter the passphrase and the backup information that you recorded earlier procedure. See Manage storage encryption using the Onboard Key Manager.

  25. Boot node into boot menu:

    boot_ontap menu.

  26. On node4, go to the boot menu and using 22/7, select the hidden option boot_after_controller_replacement. At the prompt, enter node2 to reassign the disks of node2 to node4, as per the following example.

    Expand the console output example 
    LOADER-A> boot_ontap menu
.
.
<output truncated>
.
All rights reserved.
*******************************
*                             *
* Press Ctrl-C for Boot Menu. *
*                             *
*******************************
.
<output truncated>
.
Please choose one of the following:
(1)  Normal Boot.
(2)  Boot without /etc/rc.
(3)  Change password.
(4)  Clean configuration and initialize all disks.
(5)  Maintenance mode boot.
(6)  Update flash from backup config.
(7)  Install new software first.
(8)  Reboot node.
(9)  Configure Advanced Drive Partitioning.
(10) Set Onboard Key Manager recovery secrets.
(11) Configure node for external key management.
Selection (1-11)? 22/7
(22/7)                          Print this secret List
(25/6)                          Force boot with multiple filesystem disks missing.
(25/7)                          Boot w/ disk labels forced to clean.
(29/7)                          Bypass media errors.
(44/4a)                         Zero disks if needed and create new flexible root volume.
(44/7)                          Assign all disks, Initialize all disks as SPARE, write DDR labels
.
.
<output truncated>
.
.
(wipeconfig)                        Clean all configuration on boot device
(boot_after_controller_replacement) Boot after controller upgrade
(boot_after_mcc_transition)         Boot after MCC transition
(9a)                                Unpartition all disks and remove their ownership information.
(9b)                                Clean configuration and initialize node with partitioned disks.
(9c)                                Clean configuration and initialize node with whole disks.
(9d)                                Reboot the node.
(9e)                                Return to main boot menu.
The boot device has changed. System configuration information could be lost. Use option (6) to
restore the system configuration, or option (4) to initialize all disks and setup a new system.
Normal Boot is prohibited.
Please choose one of the following:
(1)  Normal Boot.
(2)  Boot without /etc/rc.
(3)  Change password.
(4)  Clean configuration and initialize all disks.
(5)  Maintenance mode boot.
(6)  Update flash from backup config.
(7)  Install new software first.
(8)  Reboot node.
(9)  Configure Advanced Drive Partitioning.
(10) Set Onboard Key Manager recovery secrets.
(11) Configure node for external key management.
Selection (1-11)? boot_after_controller_replacement
This will replace all flash-based configuration with the last backup to disks. Are you sure
you want to continue?: yes
.
.
<output truncated>
.
.
Controller Replacement: Provide name of the node you would like to replace:
<nodename of the node being replaced>
Changing sysid of node node2 disks.
Fetched sanown old_owner_sysid = 536940063 and calculated old sys id = 536940063
Partner sysid = 4294967295, owner sysid = 536940063
.
.
<output truncated>
.
.
varfs_backup_restore: restore using /mroot/etc/varfs.tgz
varfs_backup_restore: attempting to restore /var/kmip to the boot device
varfs_backup_restore: failed to restore /var/kmip to the boot device
varfs_backup_restore: attempting to restore env file to the boot device
varfs_backup_restore: successfully restored env file to the boot device wrote
    key file "/tmp/rndc.key"
varfs_backup_restore: timeout waiting for login
varfs_backup_restore: Rebooting to load the new varfs
Terminated
<node reboots>
System rebooting...
.
.
Restoring env file from boot media...
copy_env_file:scenario = head upgrade
Successfully restored env file from boot media...
Rebooting to load the restored env file...
.
System rebooting...
.
.
.
<output truncated>
.
.
.
.
WARNING: System ID mismatch. This usually occurs when replacing a
boot device or NVRAM cards!
Override system ID? {y|n} y
.
.
.
.
Login:
    Note In the above console output example, ONTAP will prompt you for the partner node name if the system uses Advanced Disk Partitioning (ADP) disks.

  27. At the LOADER prompt, boot up:

    boot_ontap menu

    Now, on booting, the node can detect all the disks that were previously assigned to it and can boot up as expected.

    When the cluster nodes you are replacing use root volume encryption, ONTAP is unable to read the volume information from the disks. Restore the keys for the root volume:

    If the root volume is encrypted, recover the onboard key-management secrets so the system can find the root volume.

    1. Return to the special boot menu:
      LOADER> boot_ontap menu

      Please choose one of the following:
(1) Normal Boot.
(2) Boot without /etc/rc.
(3) Change password.
(4) Clean configuration and initialize all disks.
(5) Maintenance mode boot.
(6) Update flash from backup config.
(7) Install new software first.
(8) Reboot node.
(9) Configure Advanced Drive Partitioning.
(10) Set Onboard Key Manager recovery secrets.
(11) Configure node for external key management.

Selection (1-11)? 10

    2. Select (10) Set Onboard Key Manager recovery secrets

    3. Enter y at the following prompt:

      This option must be used only in disaster recovery procedures. Are you sure? (y or n): y

    4. At the prompt, enter the key-manager passphrase.

    5. Enter the backup data when prompted.

      Note You must have obtained the passphrase and backup data in the Prepare the nodes for upgrade section of this procedure.

    6. After the system boots to the special boot menu again, run option (1) Normal Boot

      Note You might encounter an error at this stage. If an error occurs, repeat the substeps in Step 27 until the system boots normally.