Skip to main content

Copy management

Contributors
PDFs

Object data is managed by the active ILM policies and associated ILM rules. ILM rules make replicated or erasure-coded copies to protect object data from loss.

Different types or locations of object copies might be required at different times in the object's life. ILM rules are periodically evaluated to ensure that objects are placed as required.

Object data is managed by the LDR service.

Content protection: replication

If an ILM rule's content placement instructions require replicated copies of object data, copies are made and stored to disk by the Storage Nodes that make up the configured storage pool.

The ILM engine in the LDR service controls replication and ensures that the correct number of copies are stored in the correct locations and for the correct amount of time.

diagram showing the process described in surrounding text

  1. The ILM engine queries the ADC service to determine the best destination LDR service within the storage pool specified by the ILM rule. It then sends that LDR service a command to initiate replication.

  2. The destination LDR service queries the ADC service for the best source location. It then sends a replication request to the source LDR service.

  3. The source LDR service sends a copy to the destination LDR service.

  4. The destination LDR service notifies the ILM engine that the object data has been stored.

  5. The ILM engine updates the metadata store with object location metadata.

Content protection: erasure coding

If an ILM rule includes instructions to make erasure-coded copies of object data, the applicable erasure-coding scheme breaks object data into data and parity fragments and distributes these fragments across the Storage Nodes configured in the erasure-coding profile.

The ILM engine, which is a component of the LDR service, controls erasure coding and ensures that the erasure-coding profile is applied to object data.

Erasure Coding Data Flow

  1. The ILM engine queries the ADC service to determine which DDS service can best perform the erasure coding operation. When determined, the ILM engine sends an "initiate" request to that service.

  2. The DDS service instructs an LDR to erasure code the object data.

  3. The source LDR service sends a copy to the LDR service selected for erasure coding.

  4. After creating the appropriate number of parity and data fragments, the LDR service distributes these fragments across the Storage Nodes (Chunk services) that make up the erasure-coding profile's storage pool.

  5. The LDR service notifies the ILM engine, confirming that object data is successfully distributed.

  6. The ILM engine updates the metadata store with object location metadata.

Content protection: Cloud Storage Pool

If an ILM rule's content placement instructions require that a replicated copy of object data is stored on a Cloud Storage Pool, object data is duplicated to the external S3 bucket or Azure Blob storage container that was specified for the Cloud Storage Pool.

The ILM engine, which is a component of the LDR service, and the Data Mover service control the movement of objects to the Cloud Storage Pool.

Cloud Storage Pool Data Flow

  1. The ILM engine selects a Data Mover service to replicate to the Cloud Storage Pool.

  2. The Data Mover service sends the object data to the Cloud Storage Pool.

  3. The Data Mover service notifies the ILM engine that the object data has been stored.

  4. The ILM engine updates the metadata store with object location metadata.