Rebalance FlexGroup volumes by moving files

02/14/2025 Contributors

PDFs

Beginning with ONTAP 9.12.1, you can rebalance FlexGroup volumes by non-disruptively moving files from one constituent in a FlexGroup to another constituent.

FlexGroup rebalancing helps redistribute capacity when imbalances develop over time due to the addition of new files and file growth. After you manually start the rebalance operation, ONTAP selects the files and moves them automatically and non-disruptively.

You should be aware that FlexGroup rebalancing degrades system performance when large numbers of files are moved as part of a single rebalancing event or over multiple rebalancing events because of the creation of multi-part inodes. Every file moved as part of a rebalancing event has 2 multi-part inodes associated with that file. The larger the number of files with multi-part inodes as a percentage of the total number of files in a FlexGroup, the greater the performance impact. Certain use cases, such as a FlexVol to FlexGroup conversion, can result in a significant amount of multi-part inode creation.

Rebalancing is available only when all nodes in the cluster are running ONTAP 9.12.1 or later releases. You must enable granular data functionality on any FlexGroup volume that runs the rebalancing operation. Once that functionality is enabled, you cannot revert to ONTAP 9.11.1 and earlier versions unless you delete this volume or restore from a snapshot that was created before the setting was enabled.

Beginning with ONTAP 9.14.1, ONTAP introduces an algorithm to non-disruptively and proactively move files in volumes that have granular data enabled without user interaction. The algorithm operates in very specific, targeted scenarios to alleviate performance bottlenecks. Scenarios where this algorithm might act include very heavy write load on a particular set of files on one node in the cluster or a continually growing file in a very hot parent directory.

Beginning with ONTAP 9.16.1, you can also enable advanced capacity balancing to redistribute a large file's data between FlexGroup member volumes.

FlexGroup rebalancing considerations

You should be aware of how FlexGroup rebalancing works and how it interacts with other ONTAP features.

FlexVol to FlexGroup conversion

It is recommended that you not use automatic FlexGroup rebalancing after a FlexVol to FlexGroup conversion. Instead, you can use the disruptive retroactive file move feature available in ONTAP 9.10.1 and later, by entering the volume rebalance file-move command. Learn more about volume rebalance file-move start in the ONTAP command reference.

Rebalancing with the automatic FlexGroup rebalancing feature can degrade performance when moving large numbers of files, like when you perform a FlexVol to FlexGroup conversion, and as much as 50 to 85% of the data on the FlexVol volume is moved to a new constituent.
Minimum and maximum file size

File selection for automatic rebalancing is based on blocks saved. The minimum file size considered for rebalancing is 100 MB by default (can be configured as low as 20 MB using the min-file-size parameter shown below) and the maximum file size is 100 GB.
Files in snapshots

You can configure FlexGroup rebalancing to only consider files to be moved which are not currently present in any snapshots. When rebalancing is started, a notification displays if a snapshot operation is scheduled anytime during a rebalancing operation.

Snapshots are restricted if a file is being moved and is undergoing framing at the destination. A snapshot restore operation is not allowed while file rebalancing is in progress.

Any snapshot created after the granular-data option is enabled cannot be replicated to a system running ONTAP 9.11.1 and earlier versions because ONTAP 9.11.1 and earlier versions do not support multi-part inodes.
SnapMirror operations

FlexGroup rebalancing should take place between scheduled SnapMirror operations. A SnapMirror operation might fail if a file is being relocated before a SnapMirror operation begins if that file move does not complete within the 24-minute SnapMirror retry period. Any new file relocation that begins after a SnapMirror transfer has started will not fail.
File-based compression storage efficiency

With file-based compression storage efficiency, the file is decompressed before it’s moved to the destination, so the compression savings is lost. The compression savings is regained after a manually initiated background scanner runs on the FlexGroup volume after rebalancing. However, if any file is associated with a snapshot on any volume, the file will be ignored for compression.
Deduplication

Moving deduplicated files can cause increased overall usage for the FlexGroup volume. During file rebalancing, only unique blocks are moved to the destination, freeing that capacity on the source. Shared blocks remain on the source and are copied to the destination. While this achieves the goal of reducing the used capacity on a nearly full source constituent, it can also lead to increased overall usage on the FlexGroup volume due to copies of shared blocks on the new destinations. This is also possible when files that are part of a snapshot are moved. The space savings is not fully recognized until the snapshot schedule recycles and there are no longer copies of the files in snapshots.
FlexClone volumes

If file rebalancing is in progress when a FlexClone volume is created, the rebalancing will not be performed on the FlexClone volume. Rebalancing on the FlexClone volume should be performed after it is created.
File move

When a file is moved during a FlexGroup rebalancing operation, the file size is reported as part of quota accounting on both the source and destination constituents. Once the move is completed, quota accounting returns to normal, and the file size is only reported on the new destination.
Autonomous Ransomware Protection

Beginning with ONTAP 9.13.1, Autonomous Ransomware Protection is supported during disruptive and non-disruptive rebalance operations.
Object store volumes

Volume capacity rebalancing is not supported on object store volumes, such as S3 buckets.

Enable FlexGroup rebalancing

Beginning with ONTAP 9.12.1, you can enable automatic non-disruptive FlexGroup volume rebalancing to redistribute files between FlexGroup constituents.

Beginning with ONTAP 9.13.1, you can schedule a single FlexGroup rebalancing operation to begin at a date and time in the future.

Before you begin

You must have enabled the granular-data option on the FlexGroup volume before enabling FlexGroup rebalancing. You can enable it by using one of these methods:

When you create FlexGroup volume using the volume create command
By modifying an existing FlexGroup volume to enable the setting using the volume modify command

Setting it automatically when FlexGroup rebalancing is initiated using the volume rebalance command

If you are using ONTAP 9.16.1 or later and FlexGroup advanced capacity balancing is enabled using either the granular-data advanced option in the ONTAP CLI or using System Manager, FlexGroup rebalancing is also enabled.

Steps

You can manage FlexGroup rebalancing by using ONTAP System Manager or the ONTAP CLI.

Navigate to Storage > Volumes and locate the FlexGroup volume to rebalance.
Select to view the volume details.
Under FlexGroup Balance Status select Rebalance.

The Rebalance option is only available when the FlexGroup status is out of balance.
In the Rebalance Volume window, change the default settings as needed.
To schedule the rebalancing operation, select Rebalance Later and enter the date and time.

Modify FlexGroup rebalance configurations

You can change a FlexGroup rebalancing configuration to update the imbalance threshold, number of concurrent files moves minimum file size, maximum runtime, and to include or exclude snapshots. Options to modify your FlexGroup rebalancing schedule are available beginning with ONTAP 9.13.1.

Navigate to Storage > Volumes and locate the FlexGroup volume to rebalance.
Select to view the volume details.
Under FlexGroup Balance Status select Rebalance.

The Rebalance option is only available when the FlexGroup status is out of balance.
In the Rebalance Volume window, change the default settings as needed.

Stop FlexGroup rebalance

After FlexGroup rebalancing is enabled or scheduled, you can stop it at any time.

Navigate to Storage > Volumes and locate the FlexGroup volume.
Select to view the volume details.
Select Stop Rebalance.

View FlexGroup rebalance status

You can display the status about a FlexGroup rebalance operation, the FlexGroup rebalance configuration, the rebalance operation time, and the rebalance instance details.

Navigate to Storage > Volumes and locate the FlexGroup volume.
Select to view the FlexGroup details.
FlexGroup Balance Status is displayed near the bottom of the details pane.
To view information about the last rebalance operation, select Last Volume Rebalance Status.