Learn about the NetApp AI Data Engine
Suggest changes
PDFs
NetApp AI Data Engine (AIDE) is a storage‑integrated AI data service that runs on data compute nodes attached to an AFX cluster. You can optionally install and integrate AIDE with your AFX cluster, extending AFX by providing dedicated, GPU‑accelerated data processing for AI and ML workloads, while retaining all the benefits of ONTAP.
AIDE overview
AIDE is a system designed to accelerate and simplify the preparation and management of your data for AI and ML workloads. It is installed separately and integrates closely with an AFX storage system cluster. AIDE leverages the high-performance and scalable AFX storage foundation while adding specialized services for AI data processing, governance, and workflow automation.
A core strength of AIDE is its ability to centralize and automate the management of metadata across large, distributed datasets. It can be configured to continuously catalog and update the metadata store based on the data in your ONTAP volumes, enabling users to search, classify, and apply governance policies efficiently. The internal processing pipeline generates AI-ready datasets, including the requisite creation of vector embeddings for semantic search and retrieval tasks.
Security and governance are also central to its design. AIDE provides authentication, access control with new RBAC roles, data classification, and policy-driven enforcement for the protection of sensitive information. Features such as automated redaction, masking, and audit logging help you meet applicable regulatory requirements and protect critical data throughout the entire AI lifecycle.
AIDE empowers data engineers and data scientists by providing tools for rapid data discovery, curation, and workflow management. With the dedicated AIDE console and enhanced AFX REST API, users can create curated data collections, perform vector searches, and integrate with external AI and GenAI applications. AIDE reduces the required manual effort and enables teams to focus on extracting insights from their data.
Installation options
There are two options for installing and integrating AIDE with your AFX storage cluster.
Set up AFX and AIDE together
Deploying the AFX storage system and AIDE simultaneously provides a unified and streamlined installation process, ensuring immediate integration and optimal configuration. This turnkey approach simplifies onboarding, guarantees compatibility, and enables you to quickly leverage a fully operational AI data platform which is ideal for new deployments seeking rapid time-to-value.
Integrate AIDE with an existing AFX cluster
If you already have an AFX storage cluster installed and operational, you can add AIDE at any time to enhance the access and value of your data. AIDE is deployed as an add-on to the existing AFX storage environment, connecting to the established cluster storage network without disrupting operations. The integration process automatically discovers available ONTAP volumes and synchronizes metadata. Your data processing workflows are enhanced through AI automation, governance, and curation tools while preserving their existing storage foundation.