Customize ONTAP FabricPool object deletion and defragmentation
Suggest changes
PDFs
FabricPool does not delete blocks from attached object stores. Instead, FabricPool deletes objects after a certain percentage of the blocks in the object are no longer referenced by ONTAP.
For example, there are 1,024 4KB blocks in a 4MB object tiered to Amazon S3. Defragmentation and deletion do not occur until less than 205 4KB blocks (20% of 1,024) are being referenced by ONTAP. When enough (1,024) blocks have zero references, their original 4MB objects are deleted, and a new object is created.
You can customize the unreclaimed space threshold percentage and set it to different default levels for different object stores. The default settings are:
Object Store |
ONTAP 9.8 and later |
ONTAP 9.7 to 9.4 |
ONTAP 9.3 and earlier |
Cloud Volumes ONTAP |
|---|---|---|---|---|
Amazon S3 |
20% |
20% |
0% |
30% |
Google Cloud Storage |
20% |
12% |
n/a |
35% |
Microsoft Azure Blob Storage |
25% |
15% |
n/a |
35% |
NetApp ONTAP S3 |
40% |
n/a |
n/a |
n/a |
NetApp StorageGRID |
40% |
40% |
0% |
n/a |
Unreclaimed space threshold
Changing the default unreclaimed space threshold settings will increase or decrease the accepted amount of object fragmentation. Reducing fragmentation will reduce the amount of physical capacity used by the cloud tier at the expense of additional object store resources (reads and writes).
Threshold reduction
To avoid additional expenses, consider reducing the unreclaimed space thresholds when using object store pricing schemes that reduce the cost of storage but increase the cost of reads. Examples include Amazon's Standard-IA and Azure Blob Storage's Cool.
For example, tiering a volume of 10-year-old projects that has been saved for legal reasons might be less expensive when using a pricing scheme such as Standard-IA or Cool than it would be when using standard pricing schemes. Although reads are more expensive for such a volume, including reads required by object defragmentation, they are unlikely to occur frequently.
Threshold increases
Alternatively, consider increasing unreclaimed space thresholds if object fragmentation causes significantly more object store capacity to be used than necessary for the data being referenced by ONTAP. For example, using an unreclaimed space threshold of 20% in a worst-case scenario where all objects are equally fragmented to the maximum allowable extent means that it is possible for 80% of total capacity in the cloud tier to be unreferenced by ONTAP. For example:
2TB referenced by ONTAP + 8TB unreferenced by ONTAP = 10TB total capacity used by the cloud tier.
In this situation, it might be advantageous to increase the unreclaimed space threshold or increase volume minimum cooling days to reduce the capacity used by unreferenced blocks.
|
As the system defragments objects and increases their storage efficiency, it might fragment the underlying files by writing referenced blocks to new, more efficient objects. If you significantly increase the unreclaimed space threshold, you can create objects that are more storage efficient but have reduced sequential read performance. This additional activity results in increased costs from third party S3 providers, such as AWS, Azure, and Google. NetApp recommends avoiding increasing the unreclaimed space threshold above 60%. |
Change the unreclaimed space threshold
You can customize the unreclaimed space threshold percentage for different object stores.
Advanced privilege level is required.
-
To change the default unreclaimed space threshold, customize and run the following command:
storage aggregate object-store modify -aggregate <name> -object-store-name <name> -unreclaimed-space-threshold <%> (0%-99%)