The NetApp AI storage architecture places data on the storage tier that best matches each lifecycle phase. StorageGRID is used for scalable, cost-optimized raw-data ingestion and governed archival. ONTAP NAS and ONTAP S3 are used as the active preparation, data mobility staging, and object-based training workflows. E-Series with LustreFS is used when high-throughput, parallel training I/O is required. Apache Airflow orchestrates the lifecycle, while NetApp XCP moves prepared data from the ONTAP NAS tier to the selected active training tier.

This workflow diagram shows the Airflow-controlled lifecycle from raw StorageGRID data through preparation, optional XCP mobility, model execution, checkpointing, and archival. enable_xcp determines whether prepared data is copied from the ONTAP NAS bucket to ONTAP S3 or LustreFS before training. The archive branches are independent: manual_archive_stage=model_training archives the baseline immediately after training, while manual_archive_stage=inferencing archives the complete outcome after predictions are generated.

This sequence diagram identifies which platform component performs each exchange. Airflow reads raw objects from StorageGRID and writes prepared data to the ONTAP NAS bucket; it then invokes XCP over SSH. XCP reads the ONTAP NAS NFS export and writes to the configured training tier. The final conditional block shows that StorageGRID archival content depends on the selected manual archive stage.

This deployment diagram shows the required hosts, storage endpoints, and network interfaces. The Airflow host uses S3 APIs for StorageGRID, the ONTAP NAS bucket, and the selected ONTAP S3 destination, and uses SSH to control the XCP host. The XCP host accesses the prepared-data source through NFS and writes to either ONTAP S3 or the LustreFS mount; when LustreFS is selected, the Airflow host must also mount it to read training data and materialize artifacts.

This compact flow summarizes data and artifact provenance. Each run receives a unique run_stamp, which keeps prepared data, XCP output, model artifacts, predictions, and archived objects traceable to the same execution. XCP destination selection governs the training-data and model-artifact tier, while manual_archive_stage governs whether StorageGRID receives the baseline training package or the full inference package.