Skip to main content
AI Data Engine

Learn about AI Data Engine

Contributors netapp-dbagwell
PDFs

AI Data Engine (AIDE) is a metadata intelligence service that gives you visibility into structured and unstructured data stored on your storage systems. AI Data Engine scans your storage systems and catalogs file system metadata so that you can identify storage optimization opportunities, investigate your data, and enforce governance policies.

What AI Data Engine does

AI Data Engine scans the data sources that you connect and builds a searchable metadata catalog that is refreshed on each scan cycle. From the catalog, you can:

  • Understand your storage. See how much data you have, who owns it, and where it is.

  • Reclaim capacity. Find cold, stale, and duplicate data, and see which data owners hold the most of it, so that you know who to contact about cleanup.

  • Find access risks. Identify directories with open permissions so that you can prioritize them for review.

  • Investigate and organize. Use metadata filters to find the data you care about, tag it, and save your criteria as a policy that you can monitor over time.

  • Act on data. Tag, archive, copy, or delete the files that you've reviewed.

  • Prepare data for AI. Let AI clients that you've authorized search your metadata and take actions on data.

Storage optimization and data lifecycle management

Use metadata to identify data that warrants review, such as files that meet your organization's cold and stale thresholds and duplicate files. Investigate the results before you decide whether to keep, tag, archive, copy, or delete the data.

Storage optimization insights point to opportunities. Capacity that you might reclaim depends on the actions that you choose to take.

Governed AI access

Governed AI access lets AI clients search your metadata through the Model Context Protocol (MCP), within boundaries that you set. You choose which data to make available by creating collections. A collection is a set of scanned data that you make available to AI clients. You control which collections a user can search by defining the Microsoft Entra ID scopes.

AI clients work with metadata only. They can't retrieve file contents.

Learn more about governed AI access.

Who uses AI Data Engine

The following examples show how different people might use AI Data Engine:

  • Storage administrators who need to review files meeting their organization's cold and stale thresholds, identify duplicate files, and evaluate storage optimization opportunities.

  • IT operations teams who need a central view of what structured and unstructured data exists across their storage environment.

  • Data or AI team members who use an AI client to search the metadata in an authorized collection and to tag the data that they want to work with.

  • IT leads who are rolling out AI tools and need to choose which collections AI clients can search and who can use them.

Where AI Data Engine runs

You access AI Data Engine through NetApp Console, under AI data hub > AI Data Engine. The scanning and catalog services run on infrastructure that you provide, either in your cloud account or in your on-premises environment. Choose an AIDE single-node deployment on a Linux VM, or an AIDE multi-node deployment on a Kubernetes cluster if your environment requires high availability at large scale.

Learn more about deployment options.

AI Data Engine scans the supported data sources that you connect. For more information, refer to Connect a data source.