Skip to main content

Monitor storage capacity

Contributors
PDFs

Monitor the total usable space available to ensure that the StorageGRID system does not run out of storage space for objects or for object metadata.

StorageGRID stores object data and object metadata separately, and reserves a specific amount of space for a distributed Cassandra database that contains object metadata. Monitor the total amount of space consumed for objects and for object metadata, as well as trends in the amount of space consumed for each. This will enable you to plan ahead for the addition of nodes and avoid any service outages.

You can view storage capacity information for the entire grid, for each site, and for each Storage Node in your StorageGRID system.

Monitor storage capacity for the entire grid

Monitor the overall storage capacity for your grid to ensure that adequate free space remains for object data and object metadata. Understanding how storage capacity changes over time can help you plan to add Storage Nodes or storage volumes before the grid's usable storage capacity is consumed.

The Grid Manager dashboard lets you quickly assess how much storage is available for the entire grid and for each data center. The Nodes page provides more detailed values for object data and object metadata.

Steps

  1. Assess how much storage is available for the entire grid and for each data center.

    1. Select Dashboard > Overview.

    2. Note the values on the Data space usage breakdown and the Metadata allowed space usage breakdown cards. Each card lists a percentage of storage usage, the capacity of used space, and the total space available or allowed by site.

      Note The summary does not include archival media.
      Data and metadata space usage breakdown

    3. Note the chart on the Storage over time card. Use the time period drop-down to help you determine how quickly storage is consumed.

      Storage over time chart

  2. Use the Nodes page for additional details on how much storage has been used and how much storage remains available on the grid for object data and object metadata.

    1. Select NODES.

    2. Select grid > Storage.

      Nodes Deployment Storage tab

    3. Position your cursor over the Storage used - object data and the Storage used - object metadata charts to see how much object storage and object metadata storage is available for the entire grid, and how much has been used over time.

      Note The total values for a site or the grid don't include nodes that have not reported metrics for at least five minutes, such as offline nodes.

  3. Plan to perform an expansion to add Storage Nodes or storage volumes before the grid's usable storage capacity is consumed.

    When planning the timing of an expansion, consider how long it will take to procure and install additional storage.

    Note If your ILM policy uses erasure coding, you might prefer to expand when existing Storage Nodes are approximately 70% full to reduce the number of nodes that must be added.

    For more information about planning a storage expansion, see the instructions for expanding StorageGRID.

Monitor storage capacity for each Storage Node

Monitor the total usable space for each Storage Node to ensure that the node has enough space for new object data.

About this task

Usable space is the amount of storage space available to store objects. The total usable space for a Storage Node is calculated by adding together the available space on all object stores within the node.

Conceptual drawing for Total usable space
Steps

  1. Select NODES > Storage Node > Storage.

    The graphs and tables for the node appear.

  2. Position your cursor over the Storage used - object data graph.

    The following values are shown:

    • Used (%): The percentage of the Total usable space that has been used for object data.

    • Used: The amount of the Total usable space that has been used for object data.

    • Replicated data: An estimate of the amount of replicated object data on this node, site, or grid.

    • Erasure-coded data: An estimate of the amount of erasure-coded object data on this node, site, or grid.

    • Total: The total amount of usable space on this node, site, or grid. The Used value is the storagegrid_storage_utilization_data_bytes metric.

      Nodes Page Storage Used Object Data

  3. Review the Available values in the Volumes and Object stores tables, below the graphs.

    Note To view graphs of these values, click the chart icons Chart Icon in the Available columns.
    Object stores tables

  4. Monitor the values over time to estimate the rate at which usable storage space is being consumed.

  5. To maintain normal system operations, add Storage Nodes, add storage volumes, or archive object data before usable space is consumed.

    When planning the timing of an expansion, consider how long it will take to procure and install additional storage.

    Note If your ILM policy uses erasure coding, you might prefer to expand when existing Storage Nodes are approximately 70% full to reduce the number of nodes that must be added.

    For more information about planning a storage expansion, see the instructions for expanding StorageGRID.

    The Low object data storage alert is triggered when insufficient space remains for storing object data on a Storage Node.

Monitor object metadata capacity for each Storage Node

Monitor the metadata usage for each Storage Node to ensure that adequate space remains available for essential database operations. You must add new Storage Nodes at each site before object metadata exceeds 100% of the allowed metadata space.

About this task

StorageGRID maintains three copies of object metadata at each site to provide redundancy and to protect object metadata from loss. The three copies are evenly distributed across all Storage Nodes at each site using the space reserved for metadata on storage volume 0 of each Storage Node.

In some cases, the grid's object metadata capacity might be consumed faster than its object storage capacity. For example, if you typically ingest large numbers of small objects, you might need to add Storage Nodes to increase metadata capacity even though sufficient object storage capacity remains.

Some of the factors that can increase metadata usage include the size and quantity of user metadata and tags, the total number of parts in a multipart upload, and the frequency of changes to ILM storage locations.

Steps

  1. Select NODES > Storage Node > Storage.

  2. Position your cursor over the Storage used - object metadata graph to see the values for a specific time.

    Storage Used - Object Metadata
    Used (%)

    The percentage of the allowed metadata space that has been used on this Storage Node.

    Prometheus metrics: storagegrid_storage_utilization_metadata_bytes and storagegrid_storage_utilization_metadata_allowed_bytes

    Used

    The bytes of the allowed metadata space that have been used on this Storage Node.

    Prometheus metric: storagegrid_storage_utilization_metadata_bytes

    Allowed

    The space allowed for object metadata on this Storage Node. To learn how this value is determine for each Storage Node, see the full description of Allowed metadata space.

    Prometheus metric: storagegrid_storage_utilization_metadata_allowed_bytes

    Actual reserved

    The actual space reserved for metadata on this Storage Node. Includes the allowed space and the required space for essential metadata operations. To learn how this value is calculated for each Storage Node, see the full description of Actual reserved space for metadata.

    Prometheus metric will be added in a future release.

    Note The total values for a site or the grid don't include nodes that have not reported metrics for at least five minutes, such as offline nodes.

  3. If the Used (%) value is 70% or higher, expand your StorageGRID system by adding Storage Nodes to each site.

    Caution The Low metadata storage alert is triggered when the Used (%) value reaches certain thresholds. Undesirable results can occur if object metadata uses more than 100% of the allowed space.

    When you add the new nodes, the system automatically rebalances object metadata across all Storage Nodes within the site. See the instructions for expanding a StorageGRID system.

Monitor space usage forecasts

Monitor space usage forecasts for user data and metadata to estimate when you will need to expand a grid.

If you notice that the rate of consumption changes over time, select a shorter range from the Averaged over pull-down to reflect only the most recent ingest patterns. If you notice seasonal patterns, select a longer range.

If you have a new StorageGRID installation, allow data and metadata to accumulate before evaluating the space usage forecasts.

Steps

  1. On the dashboard, select Storage.

  2. View the dashboard cards, Forecast of data usage by storage pool and Forecast of metadata usage by site.

  3. Use these values to estimate when you will need to add new Storage Nodes for data and metadata storage.

Forecast of metadata usage by site