Reviewing warnings about storage volume recovery

Before recovering failed storage volumes for a Storage Node, you must review the following warnings.

About this task

The storage volumes (or rangedbs) in a Storage Node are identified by a hexadecimal number from 0000 to 000F, which is known as the volume ID. The first object store (volume 0) on each Storage Node uses 3 TB of space for object metadata and Cassandra database operations; any remaining space on that volume is used for object data. All other storage volumes are used exclusively for object data.

If volume 0 fails and needs to be recovered, the Cassandra database might be rebuilt as part of the volume recovery procedure. Cassandra might also be rebuilt in the following circumstances:

  • A Storage Node is brought back online after having been offline for more than 15 days.
  • The system drive and one or more storage volumes fails and is recovered.

When Cassandra is rebuilt, the system uses information from other Storage Nodes. If too many Storage Nodes are offline, some Cassandra data might not be available. If Cassandra has been rebuilt recently, Cassandra data might not yet be consistent across the grid.

CAUTION:
Do not perform this procedure if more than one Storage Node is offline, or if more than one Storage Node has a failure of any kind. Doing so might result in data loss. Contact technical support.
CAUTION:
When you recover a failed storage volume, the script prompts you to rebuild the Cassandra database if rebuilding Cassandra is necessary. Do not rebuild Cassandra on more than one Storage Node within a 15-day period. Rebuilding Cassandra on two or more nodes within 15 days of each other may result in data loss. Contact technical support.
Attention: If ILM rules are configured to store only one replicated copy and the copy exists on a Storage Node that has failed, you will not be able to recover the object. However, you must still perform the procedure to restore object data to a storage volume to purge lost object information from the database.
Note: If you encounter a DDS alarm during recovery, see troubleshooting instructions to recover from the alarm by rebuilding Cassandra. After Cassandra is rebuilt, alarms should clear. If alarms do not clear, contact technical support.