Skip to main content
Install and maintain

Automated boot recovery - ASA A1K

Contributors netapp-jsnyder dougthomp
PDFs

You can restore image on the boot media from the partner controller using the automated boot recovery process.

Select the single node automated recovery option that matches your configuration.

Option 1: Recovery with no encryption

You can restore the ONTAP image (boot media recovery) from the partner node using the boot_recovery -partner command with ASA r2 platforms running ONTAP 9.16.0 and later.

Before you begin

When you boot a node and the boot media on that node is corrupted, you'll see the following messages and the boot process with stop at the LOADER prompt:

Can't find primary boot device u0a.0
Can't find backup boot device u0a.1
ACPI RSDP Found at 0x777fe014


Starting AUTOBOOT press Ctrl-C to abort...
Could not load fat://boot0/X86_64/freebsd/image1/kernel:Device not found

ERROR: Error booting OS on: 'boot0' file: fat://boot0/X86_64/Linux/image1/vmlinuz (boot0,fat)


ERROR: Error booting OS on: 'boot0' file: fat://boot0/X86_64/freebsd/image1/kernel (boot0,fat)


Autoboot of PRIMARY image failed. Device not found (-6)
LOADER-A>

If you see this message, you must restore the ONTAP image

Steps

  1. From the LOADER prompt, enter the boot_recovery -partner command.

    The screen will displays the message Starting boot media recovery (BMR) process press Ctrl-C to abort…​ and begins initial checks.

  2. Monitor the process as LOADER configures the local cluster ports and executes netboot through http://<remote-partner-IP>:65530/recoverydisk/image.tgz.

    Once netboot is running, Starting BMR …​ is displayed on the screen and the process completes the installation process.

    1. If Key Manager is not configured, you will see the following message:

      key manager is not configured. Exiting.

    2. If you see the following message, Onboard Key Manager (OKM) is configured:

      key manager is configured.
Entering Bootmenu Option 10...

This option must be used only in disaster recovery procedures. Are you sure? (y or n):

      Go to to complete the recovery process.

    3. If you see the following message, External Key Manager (EKM) is configured. Go to the EKM topic and complete the recovery process:

      Error when fetching key manager config from partner 169.254.139.209: 28
Has key manager been configured on this system? {y|n}

  3. Monitor the BMR process as it executes restore backup config, env file, mdb, and rdb from the partner.

  4. The node reboots and BMR is complete when you see the following:

varfs_backup_restore: update checksum for varfs.tgz
varfs_backup_restore: restore using /cfcard/x86_64/freebsd/oldvarfs.tgz
varfs_backup_restore: attempting to restore /var/kmip to the boot device
varfs_backup_restore: failed to restore /var/kmip to the boot device
varfs_backup_restore: Rebooting to load the new varfs
.
Terminated
varfs_backup_restore: bootarg.abandon_varfs is set! Skipping /var backup.
Option 2: Recovery with Onboard Key Manager present

You can restore the ONTAP image (boot media recovery) from the partner node using the boot_recovery -partner with ASA r2 platforms running ONTAP 9.16.0 and later.

Before you begin

When you boot a node and the boot media on that node is corrupted, you'll see the following messages and the boot process with stop at the LOADER prompt:

Can't find primary boot device u0a.0
Can't find backup boot device u0a.1
ACPI RSDP Found at 0x777fe014


Starting AUTOBOOT press Ctrl-C to abort...
Could not load fat://boot0/X86_64/freebsd/image1/kernel:Device not found

ERROR: Error booting OS on: 'boot0' file: fat://boot0/X86_64/Linux/image1/vmlinuz (boot0,fat)


ERROR: Error booting OS on: 'boot0' file: fat://boot0/X86_64/freebsd/image1/kernel (boot0,fat)


Autoboot of PRIMARY image failed. Device not found (-6)
LOADER-A>

If you see this message, you must restore the ONTAP image

Steps

  1. From the LOADER prompt, enter the boot_recovery -partner command.

    The screen will displays the message Starting boot media recovery (BMR) process press Ctrl-C to abort…​ and begins initial checks and installation of the boot recovery files.

    1. If Onboard Key Manager (OKM) is configured, you wil see the following displayed:

      key manager is configured.
Entering Bootmenu Option 10...

This option must be used only in disaster recovery procedures. Are you sure? (y or n):

  2. Enter y at the prompt.

  3. Enter the passphrase for onboard key manager when you see Enter the passphrase for onboard key management:

  4. Enter the pass phrase for onboard key manager again when prompted to confirm the passphrase.

    Enter the passphrase for onboard key management:
Enter the passphrase again to confirm:
Enter the backup data:
TmV0QXBwIEtleSBCbG9iAAECAAAEAAAAcAEAAAAAAAA3yR6UAAAAACEAAAAAAAAA
QAAAAAAAAACJz1u2AAAAAPX84XY5AU0p4Jcb9t8wiwOZoqyJPJ4L6/j5FHJ9yj/w
RVDO1sZB1E4HO79/zYc82nBwtiHaSPWCbkCrMWuQQDsiAAAAAAAAACgAAAAAAAAA
3WTh7gAAAAAAAAAAAAAAAAIAAAAAAAgAZJEIWvdeHr5RCAvHGclo+wAAAAAAAAAA
IgAAAAAAAAAoAAAAAAAAAEOTcR0AAAAAAAAAAAAAAAACAAAAAAAJAGr3tJA/LRzU
QRHwv+1aWvAAAAAAAAAAACQAAAAAAAAAgAAAAAAAAABHVFpxAAAAAHUgdVq0EKNp
.
.
.
.

    You will see the following when the recovery process is complete:

    Trying to recover keymanager secrets....
Setting recovery material for the onboard key manager
Recovery secrets set successfully
Trying to delete any existing km_onboard.wkeydb file.

Successfully recovered keymanager secrets.

  5. Monitor the BMR process as it executes restore backup config, env file, mdb, and rdb from the partner.

    When the restore is complete, the node reboots to complete the process.

Option 3: Recovery with External Key Manager present

You can restore the ONTAP image (boot media recovery) from the partner node using the boot_recovery -partner with ASA r2 platforms running ONTAP 9.16.0 and later.

When you boot a node and the boot media on that node is corrupted, you'll see the following messages and the boot process with stop at the LOADER prompt:

Can't find primary boot device u0a.0
Can't find backup boot device u0a.1
ACPI RSDP Found at 0x777fe014


Starting AUTOBOOT press Ctrl-C to abort...
Could not load fat://boot0/X86_64/freebsd/image1/kernel:Device not found

ERROR: Error booting OS on: 'boot0' file: fat://boot0/X86_64/Linux/image1/vmlinuz (boot0,fat)


ERROR: Error booting OS on: 'boot0' file: fat://boot0/X86_64/freebsd/image1/kernel (boot0,fat)


Autoboot of PRIMARY image failed. Device not found (-6)
LOADER-A>

If you see this message, you must restore the ONTAP image.

Steps

  1. From the LOADER prompt, enter the boot_recovery -partner command.

    The screen will displays the message Starting boot media recovery (BMR) process press Ctrl-C to abort…​ and begins initial checks and installation of the boot recovery files.

    1. If External Key Manager (EKM) is configured, you will see the following displayed:

      Error when fetching key manager config from partner 169.254.139.209: 28
Has key manager been configured on this system? {y|n}

    2. Enter y if a key manager has been configured.

      key manager is configured.
Entering Bootmenu Option 11...

    Bootmenu Option 11 will prompt the user for all of the EKM configuration information so that the configuration files can be rebuilt.

  2. Enter the EKM confiuration at each prompt.

    NOTE: Most of this information was entered when EKM was originally enabled. You should enter the same information that was entered during initial EKM configuration.

  3. Check that the Keystore UUID and Cluster UUID are correct.

    1. On the partner node retrieve the Cluster UUID with the cluster identity show command.

    2. On the partner node retrieve the Keystore UUID with the vserver show -type admin command and the key-manager keystore show -vserver <nodename> command.

    3. Enter the values for Keystore UUID and Cluster UUID when prompted.

      NOTE: If the partner node is not available, the Keystore UUID and Cluster UUID can be obtained from the Mroot-AK key located on the configured key server.

      Verify the x-NETAPP-ClusterName: <cluster name> for the Cluster UUID and x-NETAPP-KeyUsage: "MROOT-AK" for the Keystore UUID attributes to ensure you have the correct keys.

  4. Monitor the retrieve and restore of Mroot-AK into the ONTAP node.

  5. If the process cannot restore the key, you will see the following message and need to configure e0M from the menu system shell:

    ERROR: kmip_init: halting this system with encrypted mroot...
WARNING: kmip_init: authentication keys might not be available.
********************************************************
*                 A T T E N T I O N                    *
*                                                      *
*       System cannot connect to key managers.         *
*                                                      *
********************************************************
ERROR: kmip_init: halting this system with encrypted mroot...
.
Terminated

Uptime: 11m32s
System halting...

LOADER-B>

    1. Run the boot_recovery -partner command on recoverery node.

    2. When prompted to perform (y or n) the options for EKM, select n for all.

      After selecting n option for the 8 prompts, the system will stop at boot menu.

    3. Collect the /cfcard/kmip/servers.cfg file information from another cluster node. You will collect the following information:

      • The KMIP server address.

      • The KMIP port.

      • The Keystore UUID.

      • A copy of the client certificate from the /cfcard/kmip/certs/client.crt file.

      • A copy of the client key from the /cfcard/kmip/certs/client.key file.

      • A copy of the KMIP server CA(s) from the /cfcard/kmip/certs/CA.pem file.

    4. Enter systemshell from bootmenu by entering systemshell at the prompt.

    5. Configure network from the systemshell menu for e0M, netmask and gateway.

    6. Exit from menu systemshell with the exit command.

    7. You will see the boot menu. Select option 11 to continue EKM restore.

    8. Answer y to the following questions and enter the required information you previously collected when prompted:

      • Do you have a copy of the /cfcard/kmip/certs/client.crt file? {y/n}

      • Do you have a copy of the /cfcard/kmip/certs/client.key file? {y/n}

      • Do you have a copy of the /cfcard/kmip/certs/CA.pem file? {y/n}

      • Do you have a copy of the /cfcard/kmip/servers.cfg file? {y/n}

  6. If the key is restored properly, the recovery process continues and reboots the node.