Skip to main content

ONTAP storage efficiency overview

Contributors netapp-lenida netapp-aherbin netapp-pcarriga netapp-dbagwell
PDFs

Storage efficiency is the measure of how effectively a storage system utilizes available space by optimizing storage resources, minimizing wasted space, and reducing written data’s physical footprint. Higher storage efficiency enables you to store the maximum amount of data within the smallest possible space at the lowest possible cost. For example, utilizing storage efficient technologies that detect and eliminate duplicate data blocks and data blocks filled with zeros decreases the overall amount of physical storage you need and reduces your overall cost.

ONTAP offers a wide range of storage efficiency technologies that reduce the amount of physical hardware or cloud storage consumed by your data and that also yield significant improvements to system performance, including faster reads of data, faster copies of datasets, and faster VM provisioning.

ONTAP storage efficiency technologies include:

  • Thin provisioning

    Thin provisioning enables you to allocate storage in a volume or LUN as it is needed instead of reserving it in advance. This reduces the amount of physical storage you need by allowing you to over-allocate your volumes or LUNs based on potential usage without reserving space that isn’t currently being used.

  • Deduplication

    Deduplication reduces the amount of physical storage required for a volume in three distinct ways.

    • Zero block deduplication

      Zero block deduplication detects and eliminates data blocks filled with all zeros and only updates metadata. 100% of the space typically used by zero blocks is then saved. Zero block deduplication is enabled by default on all deduplicated volumes.

    • Inline deduplication

      Inline deduplication detects duplicate data blocks and replaces them with references to a unique shared block before data is written to disk. Inline deduplication speeds up VM provisioning by 20% to 30%. Depending upon your version of ONTAP and your platform, inline deduplication is available at the volume or aggregate level. It is enabled by default on AFF and ASA systems. You need to manually enable inline deduplication on FAS systems.

    • Background deduplication

      Background deduplication also detects duplicate data blocks and replaces them with references to a unique shared block, but further enhances storage efficiency by doing so after data is written to the disk. You can set up background deduplication to run when certain criteria are met on your storage system. For example, you might enable background deduplication to occur when your volume reaches 10% utilization. You can also manually trigger background deduplication or set it to run on a specific schedule. It is enabled by default on AFF and ASA systems. You need to manually enable background deduplication on FAS systems.

    Deduplication is supported within volumes and across volumes within an aggregate. Reads of deduplicated data typically incur no performance charge.

  • Compression

    Compression reduces the amount of physical storage required for a volume by combining data blocks in compression groups, each of which is stored as a single block. When a read or an overwrite request is received, only a small group of blocks is read, not the entire file. This process optimizes read and overwrite performance and enables greater scalability in the size of the files being compressed.

    Compression can be run inline or postprocess. Inline compression provides immediate space savings by compressing data in memory before it is written to disk. Postprocess compression first writes the blocks to disk as uncompressed and then at a scheduled time compresses the data. It is enabled by default on AFA systems. You need to manually enable compression on all other systems.

  • Compaction

    Compaction reduces the amount of physical storage required for a volume by taking data chunks that are stored in 4 KB blocks, but that are less than 4 KB in size, and combining them into a single block. Compaction takes place while data is still in memory so unnecessary space is never consumed on disks. It is enabled by default on AFF and ASA systems. You need to manually enable compaction on FAS systems.

  • FlexClone volumes, files and LUNs

    FlexClone technology leverages Snapshot metadata to create writable, point-in-time copies of a volume, file or LUN. Copies share data blocks with their parents, consuming no storage except what is required for metadata until changes are written to a copy or its parent. When a change is written, only the delta is stored.

    Where traditional dataset copies can take minutes or even hours to create, FlexClone technology lets you copy even the largest datasets almost instantaneously.

  • Temperature-sensitive storage efficiency

    ONTAP provides temperature-sensitive storage efficiency benefits by assessing how often your volume's data is accessed and mapping that frequency to the degree of compression applied to that data. For cold data that is accessed infrequently, larger data blocks are compressed. For hot data that is accessed frequently and is overwritten more often, smaller data blocks are compressed, making the process more efficient.

    Temperature-sensitive storage efficiency (TSSE), introduced in ONTAP 9.8, is enabled automatically on newly created thinly provisioned AFF volumes. It is not enabled on AFF A70, AFF A90, and AFF A1K platforms that are introduced in ONTAP 9.15.1, which use a hardware offload processor.

  • CPU or dedicated offload processor storage efficiency

    Beginning with ONTAP 9.15.1, ONTAP provides CPU or dedicated offload processor storage efficiency and data compaction on AFF A70, AFF A90, AFF A1K, FAS70, and FAS90 platforms. On AFF A70, AFF A90, and AFF A1K systems, storage efficiency is enabled automatically and requires no configuration.

You can realize the benefit of these technologies in your day-to-day operations with minimal effort. For example, suppose you need to supply 5,000 users with storage for home directories, and you estimate that the maximum space needed by any user is 1 GB. You could reserve a 5 TB aggregate in advance to meet the total potential storage need. However, you also know that home directory capacity requirements vary greatly across your organization. Instead of reserving 5 TB of total space for your organization, you can create a 2 TB aggregate. Then you can use thin provisioning to nominally assign 1 GB of storage to each user but allocate the storage only as needed. You can actively monitor the aggregate over time and increase the actual physical size as necessary.

In another example, suppose you are using a Virtual Desktop Infrastructure (VDI) with a large amount of duplicate data among your virtual desktops. Deduplication reduces your storage usage by automatically eliminating duplicate blocks of information across the VDI, replacing them with a pointer to the original block. Other ONTAP storage efficiency technologies, such as compression, can also run in the background without your intervention.

ONTAP disk partitioning technology delivers greater storage efficiency as well. RAID DP technology protects against double disk failure without sacrificing performance or adding disk-mirroring overhead. Advanced SSD partitioning with ONTAP 9 increases usable capacity by almost 20%.

NetApp provides the same storage efficiency features available with on-premises ONTAP in the cloud. When you migrate data from on-premises ONTAP to the cloud, the existing storage efficiency is preserved. For example, suppose you have an SQL database containing business-critical data that you want to move from an on-premises system to the cloud. You can use data replication in BlueXP to migrate your data and, as part of the migration process, you can enable your latest on-premises policy for Snapshot copies in the cloud.