Skip to main content
Install and maintain

Automated boot media recovery from the partner node - AFF A70 and AFF A90

Contributors netapp-lisa
PDFs

After installing the new boot media device in your AFF A70 or AFF A90 storage system, you can start the automated boot media recovery process to restore the configuration from the partner node. During the recovery process, the system checks whether encryption is enabled and determines the type of key encryption in use. If key encryption is enabled, the system guides you through the appropriate steps to restore it.

The automated boot media recovery process is supported only in ONTAP 9.17.1 and later. If your storage system is running an earlier version of ONTAP, use the manual boot recovery procedure.

Before you begin

  • Determine your key manager type:

    • Onboard Key Manager (OKM): Requires cluster-wide passphrase and backup data

    • External Key Manager (EKM): Requires the following files from the partner node:

      • /cfcard/kmip/servers.cfg

      • /cfcard/kmip/certs/client.crt

      • /cfcard/kmip/certs/client.key

      • /cfcard/kmip/certs/CA.pem

Steps

  1. From the LOADER prompt, start the boot media recovery process:

    boot_recovery -partner

    The screen displays the following message:

    Starting boot media recovery (BMR) process. Press Ctrl-C to abort…

  2. Monitor the boot media install recovery process.

    The process completes and displays the Installation complete message.

  3. The system checks for encryption and displays one of the following messages:

    If you see this message…​ Do this…​

    key manager is not configured. Exiting.

    Encryption is not installed on the system.

    1. Wait for the login prompt to display.

    2. Log into the node and give back the storage:

      storage failover giveback -ofnode impaired_node_name

    3. Go to step 6 to reenable automatic giveback if it was disabled.

    key manager is configured.

    Encryption is installed. Go to step 4 to restore the key manager.
    Note If the system cannot identify the key manager configuration, it displays an error message and prompts you to confirm whether key manager is configured and which type (onboard or external). Answer the prompts to proceed.

  4. Restore the key manager using the appropriate procedure for your configuration:

    Onboard Key Manager (OKM)

    The system displays the following message and begins running BootMenu Option 10:

    key manager is configured.
Entering Bootmenu Option 10...

This option must be used only in disaster recovery procedures. Are you sure? (y or n):

    1. Enter y at the prompt to confirm you want to start the OKM recovery process.

    2. Enter the passphrase for onboard key management when prompted.

    3. Enter the passphrase again when prompted to confirm.

    4. Enter the backup data for onboard key manager when prompted.

      Show example of passphrase and backup data prompts 
      Enter the passphrase for onboard key management:
-----BEGIN PASSPHRASE-----
<passphrase_value>
-----END PASSPHRASE-----
Enter the passphrase again to confirm:
-----BEGIN PASSPHRASE-----
<passphrase_value>
-----END PASSPHRASE-----
Enter the backup data:
-----BEGIN BACKUP-----
<passphrase_value>
-----END BACKUP-----

    5. Monitor the recovery process as it restores the appropriate files from the partner node.

      When the recovery process is complete, the node reboots. The following messages indicate a successful recovery:

      Trying to recover keymanager secrets....
Setting recovery material for the onboard key manager
Recovery secrets set successfully
Trying to delete any existing km_onboard.keydb file.

Successfully recovered keymanager secrets.

    6. After the node reboots, verify that the system is back online and operational.

    7. Return the impaired controller to normal operation by giving back its storage:

      storage failover giveback -ofnode impaired_node_name

    8. After the partner node is fully up and serving data, synchronize the OKM keys across the cluster:

      security key-manager onboard sync

      Go to step 5 to reenable automatic giveback if it was disabled.

    External Key Manager (EKM)

    The system displays the following message and begins running BootMenu Option 11:

    key manager is configured.
Entering Bootmenu Option 11...

    1. Enter the EKM configuration settings when prompted:

      1. Enter the client certificate contents from the /cfcard/kmip/certs/client.crt file:

        Show example of client certificate contents 
        -----BEGIN CERTIFICATE-----
<certificate_value>
-----END CERTIFICATE-----

      2. Enter the client key file contents from the /cfcard/kmip/certs/client.key file:

        Show example of client key file contents 
        -----BEGIN RSA PRIVATE KEY-----
<key_value>
-----END RSA PRIVATE KEY-----

      3. Enter the KMIP server CA(s) file contents from the /cfcard/kmip/certs/CA.pem file:

        Show example of KMIP server file contents 
        -----BEGIN CERTIFICATE-----
<KMIP_certificate_CA_value>
-----END CERTIFICATE-----

      4. Enter the server configuration file contents from the /cfcard/kmip/servers.cfg file:

        Show example of server configuration file contents 
        xxx.xxx.xxx.xxx:5696.host=xxx.xxx.xxx.xxx
xxx.xxx.xxx.xxx:5696.port=5696
xxx.xxx.xxx.xxx:5696.trusted_file=/cfcard/kmip/certs/CA.pem
xxx.xxx.xxx.xxx:5696.protocol=KMIP1_4
1xxx.xxx.xxx.xxx:5696.timeout=25
xxx.xxx.xxx.xxx:5696.nbio=1
xxx.xxx.xxx.xxx:5696.cert_file=/cfcard/kmip/certs/client.crt
xxx.xxx.xxx.xxx:5696.key_file=/cfcard/kmip/certs/client.key
xxx.xxx.xxx.xxx:5696.ciphers="TLSv1.2:kRSA:!CAMELLIA:!IDEA:!RC2:!RC4:!SEED:!eNULL:!aNULL"
xxx.xxx.xxx.xxx:5696.verify=true
xxx.xxx.xxx.xxx:5696.netapp_keystore_uuid=<id_value>

      5. If prompted, enter the ONTAP Cluster UUID from the partner node. You can check the cluster UUID from the partner node using the cluster identify show command.

        Show example of ONTAP Cluster UUID prompt 
        Notice: bootarg.mgwd.cluster_uuid is not set or is empty.
Do you know the ONTAP Cluster UUID? {y/n} y
Enter the ONTAP Cluster UUID: <cluster_uuid_value>


System is ready to utilize external key manager(s).

      6. If prompted, enter the temporary network interface and settings for the node:

        • The IP address for the port

        • The netmask for the port

        • The IP address of the default gateway

          Show example of temporary network setting prompts 
          In order to recover key information, a temporary network interface needs to be
configured.

Select the network port you want to use (for example, 'e0a')
e0M

Enter the IP address for port : xxx.xxx.xxx.xxx
Enter the netmask for port : xxx.xxx.xxx.xxx
Enter IP address of default gateway: xxx.xxx.xxx.xxx
Trying to recover keys from key servers....
[discover_versions]
[status=SUCCESS reason= message=]

    2. Verify the key restoration status:

      • If you see kmip2_client: Successfully imported the keys from external key server: xxx.xxx.xxx.xxx:5696 in the output, the EKM configuration has been successfully restored. The process restores the appropriate files from the partner node and reboots the node. Proceed to the next step.

      • If the key is not successfully restored, the system halts and displays error and warning messages. Rerun the recovery process from the LOADER prompt: boot_recovery -partner

        Show example of key recovery error and warning messages 
        ERROR: kmip_init: halting this system with encrypted mroot...
WARNING: kmip_init: authentication keys might not be available.
********************************************************
*                 A T T E N T I O N                    *
*                                                      *
*       System cannot connect to key managers.         *
*                                                      *
********************************************************
ERROR: kmip_init: halting this system with encrypted mroot...
.
Terminated

Uptime: 11m32s
System halting...

LOADER-B>

    3. After the node reboots, verify that the system is back online and operational.

    4. Return the controller to normal operation by giving back its storage:

      storage failover giveback -ofnode impaired_node_name

      Go to step 5 to reenable automatic giveback if it was disabled.

  1. If automatic giveback was disabled, reenable it:

    storage failover modify -node local -auto-giveback true

  2. If AutoSupport is enabled, restore automatic case creation:

    system node autosupport invoke -node * -type all -message MAINT=END

What's next

After you've restored the ONTAP image and the node is up and serving data, you return the failed part to NetApp.