Skip to main content
AI Data Engine

Assess storage health with the AI Data Engine dashboard

Contributors netapp-dbagwell
PDFs

Before you can decide what to investigate, you need to understand the state of your storage. How much storage meets your organization's cold and stale classification thresholds, and which systems or owners hold the most data in these categories? For example, a storage administrator responding to a capacity alert can see that a large portion of scanned storage is cold data, then drill into the highest-impact system to start investigating immediately.

From NetApp Console, select AI data hub > AI Data Engine. The dashboard opens by default. The dashboard organizes its insights into three tabs: Storage optimization, Permissions, and Structured data.

AI Data Engine dashboard overview

Understand how much storage is affected

Understand how much of your scanned storage is cold, stale, or exposed. The summary panel shows the scale of the problem across your scanned environment:

  • Data resources: The count of volumes, shares, and buckets currently scanned.

  • Items scanned: The total number of items indexed across all scanned data resources.

  • Cold data: Storage not accessed within the cold data threshold (default: one year). Your organization can evaluate this data according to your retention and governance policies.

  • Stale data: Storage that hasn't changed within the stale data threshold (default: three years). Stale data could warrant review according to your organization's data lifecycle policies before deciding on actions such as archival or deletion.

  • Exposed directories: The number of directories with overly permissive access settings and their associated file count. These represent a governance risk independent of storage consumption.

The cold and stale thresholds reflect your configured settings, including the timestamp and age that you choose. The default values are product defaults and don't represent legal, compliance, or industry-standard retention periods. To change them, refer to Configure data classification thresholds and access scanning.

The System and System type filters control dashboard scope. If a storage system doesn't appear in the filter list, refer to Connect a data source.

The Last updated timestamp shows when the dashboard data was last refreshed. Select Refresh to update the dashboard. To confirm that the underlying data is current, refer to View scan status and details.

Identify storage optimization opportunities

Find where you can reclaim the most storage. The Storage optimization tab includes the following charts and tables:

  • Active versus reclaimable storage: Shows the proportion of active data compared to cold and stale data across all scanned resources. A high proportion of cold, stale, or duplicated data could indicate a significant storage optimization opportunity.

  • Top data resources by reclaimable storage: Shows the data resources with the most reclaimable capacity.

  • Top consumers by reclaimable storage: Shows data owners, their reclaimable storage, and their file counts.

  • Duplicate data size distribution: Shows how duplicate data is distributed across size ranges.

  • Age of assets: Shows items by age. Use the dropdown to select the timestamp criterion for the chart.

  • Top file types: Shows the most common file types.

Use the summary panel and these charts to identify and prioritize storage optimization opportunities:

  • Focus on resources that have the largest potentially reclaimable size, rather than those with the highest total size. A small system that is mostly cold or stale data might be a better first resource to evaluate than a large system that is mostly active.

  • Look for large concentrations of cold or stale data rather than isolated files. Tens of terabytes of data that meet your stale threshold on one resource warrant evaluation. Isolated files might not require action.

Review permissions

Find the directories to prioritize for a permissions review. The Permissions tab shows the Directories with open permissions chart, which lists directory paths and their file counts. To see these directories in detail, refer to Investigate directories. To control whether AI Data Engine scans permissions, refer to Configure data classification thresholds and access scanning.

Explore structured data

See which schemas and tables are in your scanned data and compare their row counts. The Structured data tab shows the Structured data summary table, which lists the schema, database table name, and number of rows. To search and filter this data, refer to Investigate structured data.

Move from the dashboard to investigation

When you have identified a resource or owner worth investigating, use the tables on the Storage optimization tab to transition directly into the details.

The Top data resources by reclaimable storage table lists your scanned storage resources ranked by the proportion of data that is duplicated or meets your organization's cold and stale thresholds. Use the Unstructured tab to identify cold or stale files, or the Directories tab to audit permission exposure.

The Top consumers by reclaimable storage table shows which data owners hold potentially reclaimable data. Select Investigate to open the Investigation view filtered to that owner's files. From there, you can review results, act on them, or save them as a policy.

Reviewing the dashboard doesn't archive or delete data or change directory permissions.