Skip to main content
A newer release of this product is available.

Creating an Erasure Coding profile

Contributors
PDFs

To create an Erasure Coding profile, you associate a storage pool containing Storage Nodes with an erasure-coding scheme. This association determines the number of data and parity fragments created and where the system distributes these fragments.

What you'll need

  • You must be signed in to the Grid Manager using a supported browser.

  • You must have specific access permissions.

  • You must have created a storage pool that includes exactly one site or a storage pool that includes three or more sites. No erasure-coding schemes are available for a storage pool that has only two sites.

About this task

The storage pools used in Erasure Coding profiles must include exactly one site or three or more sites. If you want to provide site redundancy, the storage pool must have at least three sites.

Note You must select a storage pool that contains Storage Nodes. You cannot use Archive Nodes for erasure-coded data.
Steps

  1. Select ILM > Erasure Coding.

    The Erasure Coding Profiles page appears.

    Erasure Coding Profiles Page

  2. Click Create.

    The Create EC Profile dialog box appears.

    Create EC Profile page

  3. Enter a unique name for the Erasure Coding profile.

    Erasure Coding profile names must be unique. A validation error occurs if you use the name of an existing profile, even if that profile has been deactivated.

    Note The Erasure Coding profile name is appended to the storage pool name in the placement instruction for an ILM rule.
    Storage pool and EC profile name

  4. Select the storage pool you created for this Erasure Coding profile.

    Note If your grid currently includes only one site, you are prevented from using the default storage pool, All Storage Nodes, or any storage pool that includes the default site, All Sites. This behavior prevents the Erasure Coding profile from becoming invalid if a second site is added.
    Note If a storage pool includes exactly two sites, you cannot use that storage pool for erasure coding. No erasure-coding schemes are available for a storage pool that has two sites.

    When you select a storage pool, the list of available erasure-coding schemes is shown, based on the number of Storage Nodes and sites in the pool.

    Create EC Profile Three Sites

    The following information is listed for each available erasure-coding scheme:

    • Erasure Code: The name of the erasure-coding scheme in the following format: data fragments + parity fragments.

    • Storage Overhead (%): The additional storage required for parity fragments relative to the object's data size. Storage Overhead = Total number of parity fragments / Total number of data fragments.

    • Storage Node Redundancy: The number of Storage Nodes that can be lost while still maintaining the ability to retrieve object data.

    • Site Redundancy: Whether the selected erasure code allows the object data to be retrieved if a site is lost.

      To support site redundancy, the selected storage pool must include multiple sites, each with enough Storage Nodes to allow any site to be lost. For example, to support site redundancy using a 6+3 erasure-coding scheme, the selected storage pool must include at least three sites with at least three Storage Nodes at each site.

    Messages are displayed in these cases:

    • The storage pool you selected does not provide site redundancy. The following message is expected when the selected storage pool includes only one site. You can use this Erasure Coding profile in ILM rules to protect against node failures.

      Create EC Profile No Site Redundancy

    • The storage pool you selected does not satisfy the requirements for any erasure-coding scheme. For example, the following message is expected when the selected storage pool includes exactly two sites. If you want to use erasure coding to protect object data, you must select a storage pool with exactly one site or a storage pool with three or more sites.

      Erasure Coding Profile Error

    • Your grid includes only one site and you selected the default storage pool, All Storage Nodes, or any storage pool that includes the default site, All Sites.

      ILM EC Profile All Storage Nodes Warning

    • The erasure-coding scheme and storage pool you selected overlap with another Erasure Coding profile.

      ILM EC Profile Erasure Coding Scheme Warning

      In this example, a warning message appears because another Erasure Coding profile is using the 2+1 scheme and the storage pool for the other profile also uses one of the sites in the All 3 Sites storage pool.

      While you are not prevented from creating this new profile, you must be very careful when you start using it in the ILM policy. If this new profile is applied to existing erasure-coded objects already protected by the other profile, StorageGRID will create an entirely new set of object fragments. It will not reuse the existing 2+1 fragments. Resource issues might occur when you migrate from one Erasure Coding profile to the other, even though the erasure-coding schemes are the same.

  5. If more than one erasure-coding scheme is listed, select the one you want to use.

    When deciding which erasure-coding scheme to use, you should balance fault tolerance (achieved by having more parity segments) against the network traffic requirements for repairs (more fragments equals more network traffic). For example, when deciding between a 4+2 scheme and 6+3 scheme, select the 6+3 scheme if additional parity and fault tolerance are required. Select the 4+2 scheme if network resources are constrained to reduce network usage during node repairs.

  6. Click Save.