Reviewing warnings about storage volume recovery

Before recovering failed storage volumes for a Storage Node, you must review the following warnings.

About this task

The storage volumes (or rangedbs) in a Storage Node are identified by a hexadecimal number from 0000 to 000F, which is known as the volume ID. By default, 2 TB of space is reserved in the first object store (volume 0) for object metadata in a Cassandra database; any remaining space on that volume is used for object data. All other storage volumes are used exclusively for object data, which includes replicated copies and erasure coded fragments.

If storage volume 0 fails and needs to be recovered, the Cassandra database may be rebuilt as part of the volume recovery procedure. Cassandra might also be rebuilt in the following circumstances:

  • A Storage Node is brought back online after having been offline for more than 15 days.
  • The system drive and one or more storage volumes fails and is recovered.

When Cassandra is rebuilt, it uses information from other Storage Nodes. If too many Storage Nodes are offline, some Cassandra data may not be available. If Cassandra has been rebuilt recently, Cassandra data may not yet be consistent across the grid.

CAUTION:
Do perform this procedure if more than one Storage Node is offline, or if more than one Storage Node has a failure of any kind. Doing so might result in data loss. Contact technical support.
CAUTION:
When you recover a failed storage volume, the script will prompt you to rebuild the Cassandra database if rebuilding Cassandra is necessary. Do not rebuild Cassandra on more than one Storage Node within a 15 day period. Rebuilding Cassandra on two or more nodes within 15 days of each other may result in data loss. Contact technical support.
Attention: If the ILM policy is configured to replicate a single copy only, and those objects are lost, they cannot be recovered. You must still perform the "Restoring object data to a storage volume" procedure to purge lost object information from the database. For more information about ILM policies, see the Administrator Guide.
Note: If you encounter a DDS alarm during recovery, see the Troubleshooting Guide for instructions to recover from the alarm by rebuilding Cassandra. After Cassandra is rebuilt, alarms should clear. If alarms do not clear, contact technical support.