What erasure coding is

Erasure coding is the second method used by StorageGRID Webscale to store object data. When StorageGRID Webscale matches objects to an ILM rule that is configured to create erasure-coded copies, it slices object data into data fragments, computes additional parity fragments, and stores each fragment on a different Storage Node. When an object is accessed, it is reassembled using the stored fragments. If a data or a parity fragment becomes corrupt or lost, the erasure-coding algorithm can recreate that fragment using a subset of the remaining data and parity fragments.

The figure illustrates the use of an erasure-coding algorithm on an object’s data. In this example, the ILM rule uses a 6+3 erasure coding scheme. Each object is sliced into six equal data fragments, and three parity fragments are computed from the object data. Each of the nine fragments is stored on a different node across multiple sites to provide data protection for node failures or site loss.

Example of 6+3 erasure coding

In the example, the object can be retrieved using any six of the nine fragments. Up to three fragments can be lost without loss of the object data. If an entire data center site is lost, the object can still be retrieved or repaired, as long as all of the other fragments remain accessible.