Skip to main content

Replace VMware node in StorageGRID with SG1000 or SG100

Contributors netapp-pcarriga
PDFs

If the primary Admin Node runs as a VMware node, you can replace it with an SG1000 or SG100 services appliance.

Caution After completing and verifying the node replacement, you must delete the original VMware node. Leaving the old node in place risks a major grid disruption if it comes back online.
Before you begin

  • You have determined that the virtual machine can't be restored and must be replaced.

  • You have verified that there are no grid maintenance procedures in progress.

  • You have confirmed that the StorageGRID Appliance Installer version on the SG1000 or SG100 services appliance matches the software version of your StorageGRID system.

  • You have the following information about the existing primary Admin Node:

    Node name: You must install the services appliance using the same node name as the existing primary Admin Node. The node name is the hostname (system name).

    IP address: You can assign the services appliance the same IP address as the existing primary Admin Node (recommended), or you can select a new unused IP address on each network.

  • You have the provisioning passphrase.

Step 1: Replace the VMware node

You first remove the existing primary Admin Node in the Grid Manager. Then, you install the SG1000 or SG100 services appliance as the new primary Admin Node using the StorageGRID Appliance Installer and recover the node. If you're using a new IP address, you can install the services appliance before you remove the existing primary Admin Node.

Use existing IP address

Replace the primary Admin Node using the existing primary Admin Node IP address.

Steps

  1. Download the recovery package from the Grid Manager:

    1. Select Maintenance > Recovery package.

    2. Enter the provisioning passphrase and download the package.

  2. Log in to the existing primary Admin Node:

    1. Enter the following command: ssh admin@grid_node_IP

    2. Enter the password listed in the Passwords.txt file.

    3. Switch to root: su -

    4. Enter the password listed in the Passwords.txt file.

    When you are logged in as root, the prompt changes from $ to #.

  3. Copy the audit logs from the existing primary Admin Node.

  4. Stop Server Manager and all services on the grid node:

    shutdown -h now

    Allow up to 15 minutes for the services to stop.

  5. Install the SG1000 or SG100 services appliance.

  6. Deploy the SG1000 or SG100 services appliance.

  7. After selecing Start installation, allow approximately ten minutes for the installation to reach the Load StorageGRID installer step.

    After six to seven minutes, the browser redirects to start the recovery.

  8. Check that the appliance clock is correct.

    1. Use SSH to connect to port 8022:

      1. Enter the command: ssh -p 8022 admin@grid_node_IP.

      2. Switch to root: su -

      3. Enter netapp1!.

    2. Verify that the appliance clock is within a few minutes of the current UTC. If required, set the correct date and time:

      date -s "DD Mon YYYY HH:MM:SS"

  9. Return to the web browser and select Recover a failed primary Admin Node.

  10. Upload the recovery package you downloaded.

  11. Enter the provisioning passphrase and select Start Recovery.

    The recovery progress bar appears. Allow six to ten minutes for the recovery process to complete and the services to start.

  12. Log in to Grid Manager when prompted.

Use new IP address

Replace the primary Admin Node using a new IP address.

Steps

  1. Install the SG1000 or SG100 services appliance.

  2. Check that the appliance clock is correct to avoid NTP errors during recovery.

    1. Use SSH to connect to port 8022:

      1. Enter the command: ssh -p 8022 admin@grid_node_IP.

      2. Switch to root: su -

      3. Enter netapp1!.

    2. Verify that the appliance clock is within a few minutes of the current Coordinated Universal Time (UTC). If required, set the correct date and time:

      date -s "DD Mon YYYY HH:MM:SS"

  3. Go to Upload StorageGRID Software in the StorageGRID Appliance Installer and check that the preloaded software version on the appliance matches the StorageGRID version running on the grid. If the versions don't match:

    1. Download the correct StorageGRID Debian installation package from the NetApp Support Site.

    2. Select Remove to remove the installation software.

    3. Upload the storagegrid-webscale-images-<sg-version>.deb and storagegrid-webscale-images-<sgversion>.deb.md5 files.

  4. On the StorageGRID Appliance Installer Home tab, select Primary Admin (with Load Balancer) as Node type.

  5. Enter the node name. The node name must match the existing primary Admin Node name.

    Important Don't select Start Installation.

  6. Check if the subnet of the new IP address is in the grid network. If the new IP subnet isn't present, add it to the grid network:

    1. In Grid Manager, select Maintenance > Grid Network.

    2. Enter the provisioning passphrase to update the subnet list.

  7. Download the recovery package from GRID Manager:

    1. Select Maintenance > Recovery package.

    2. Enter the provisioning passphrase, then download the package.

  8. Log in to the existing primary Admin Node:

    1. Enter the following command: ssh admin@grid_node_IP

    2. Enter the password listed in the Passwords.txt file.

    3. Enter the following command to switch to root: su -

    4. Enter the password listed in the Passwords.txt file.

    When you are logged in as root, the prompt changes from $ to #.

  9. Copy the audit logs from the existing Primary Admin Node.

  10. Stop Server Manager and all services on the grid node:

    service servermanager stop

    Allow up to 15 minutes for the services to stop.

  11. On the StorageGRID Appliance Installer Home tab:

    1. Verify the following settings:

      • Node type is set to Primary Admin (with Load Balancer)

      • Node name matches the existing Primary Admin Node name

    2. Record the new IP address.

  12. Select Start Installation.

    Allow approximately 10 minutes for the installation to reach the Load StorageGRID installer step.

    After six to seven minutes, the browser redirects to begin the recovery procedure.

  13. Check that the appliance clock is correct.

    1. Use SSH to connect to port 8022:

      1. Enter the command: ssh -p 8022 admin@grid_node_IP.

      2. Switch to root: su -

      3. Enter netapp1!.

    2. Verify that the appliance clock is within a few minutes of the current UTC. If required, set the correct date and time:

      date -s "DD Mon YYYY HH:MM:SS"

  14. Return to the web browser, then select Recover a failed primary Admin Node.

  15. Upload the recovery package you downloaded.

  16. Enter the provisioning passphrase and select Start Recovery.

    The Primary Admin Node Recovery page disappears, then reappears. If you select Start Recovery again, a 500: Internal Server Error appears because the recovery has started but can't send the update to the Grid Manager, which is using the new IP address. Resolve this issue by updating the primary Admin Node IP address in the /etc/hosts file:

    1. Use SSH to connect to port 8022:

      1. Enter the command: ssh -p 8022 admin@grid_node_IP.

      2. Switch to root: su -

      3. Enter the password listed in the Passwords.txt file.

    2. Access the log:

      tail /var/local/log/install.log

      You should see the following error repeatedly: curl: (7) Failed to connect to <primary_Admin_Node_name> port9999: No route to host

      This is the expected behavior. StorageGRID is attempting to send the update to the primary Admin Node using the existing primary Admin Node IP address.

    3. Verify that the container is running:

      storagegrid node status

      storagegrid node enter <primary_Admin_Node_name>

    4. In the /etc/hosts file inside the container, replace the existing primary Admin Node IP address with the new IP address.

    5. Save the file.

    6. Return to the web browser. After a few seconds, the recovery progress bar appears.

      Allow six to ten minutes for the recovery process to complete and the services to start.

  17. Log in to Grid Manager when prompted.

  18. Update the recovery package:

    1. Log in to the primary Admin Node:

      1. Enter the following command: ssh admin@primary_Admin_Node_IP

      2. Switch to root: su -

      3. Enter the following command: change-ip

    2. Select option 3 – Edit admin network subnet lists, then add the new subnet.

    StorageGRID automatically generates a new recovery package with the new primary Admin Node IP address.

  19. Download the latest recovery package from the Grid Manager.

Step 2: Complete node recovery

Use an existing non-primary Admin Node as the source Admin Node for the restore procedures (audit logs, Admin Node database, and Prometheus metrics). If the grid has no non-primary Admin Node, you can use the original VMware node that is being replaced, provided it is running on a different IP address than the recovered primary Admin Node.

Steps

  1. Restore the audit log.

  2. Restore the Admin Node database.

  3. Restore Prometheus metrics.

  4. Verify that the new primary Admin Node is functioning correctly. Then, delete the original VM node or bare-metal container immediately.