Skip to main content
NetApp Solutions

Solution Overview

Contributors nkarthik
PDFs

The Hybrid Iceberg Lakehouse solution provides unique benefits to address customer challenges faced by data lake customers. By Leveraging the Dremio Unified Lakehouse platform and NetApp ONTAP, StorageGRID, and NetApp Cloud solutions, companies can add significant value to their business operations. The solution not only provides access to multiple data sources, including NetApp sources, but also enhances overall analytical performance and helps companies drive business insights that leads to business growth.

NetApp Overview

  • NetApp's offerings, such as ONTAP and StorageGRID, enable the separation of storage and computing, enabling optimal resource utilization based on specific requirements. This flexibility empowers customers to independently scale their storage using NetApp storage solutions

  • By leveraging NetApp's storage controllers, customers can efficiently serve data to their vector database using NFS and S3 protocols. These protocols facilitate customer data storage and manage the vector database index, eliminating the need for multiple copies of data accessed through file and object methods.

  • NetApp ONTAP provides native support for NAS and Object storage across leading cloud service providers like AWS, Azure, and Google Cloud. This broad compatibility ensures seamless integration, enabling customer data mobility, global accessibility, disaster recovery, dynamic scalability, and high performance.

StorageGRID

Our industry-leading object storage storageGRID offers a powerful policy engine for automated data placement, flexible deployment options, and unmatched durability with layered erasure coding. It has a scalable architecture supporting billions of objects and petabytes of data in a single namespace. The solution enables hybrid cloud integration, allowing data tiering to major cloud platforms. It has been recognized as a leader in the 2019 IDC Marketscape Worldwide Object-Based Vendor Assessment.

Additionally, storageGRID excels in managing unstructured data at scale with software-defined object storage, geo-redundancy, and multi-site capabilities. It incorporates policy-based information lifecycle management and offers cloud integration features like mirroring and search. It has various certifications, including Common Criteria, NF203 Digital Safe Component, ISO/IEC 25051, KPMG, and Cohasset Compliance Assessment.

In summary, NetApp storageGRID provides powerful features, scalability, hybrid cloud integration, and compliance certifications for efficient management of unstructured data at scale.

NetApp ONTAP

NetApp ONTAP is a robust storage solution that offers a wide range of enterprise features. It includes Snapshot, which provides application-consistent and tamper-proof instant backups. SnapRestore enables near-instant restore of backups on demand, while SnapMirror offers integrated remote backup and disaster recovery capabilities. The solution also incorporates Autonomous Ransomware Protection (ARP), ensuring data security with features like multi-administrator verification, data-at-rest encryption with FIPS certification, in-transit data encryption, multifactor authentication (MFA), and role-based access control (RBAC). Comprehensive logging, auditing, onboard and external key management, secure purge, and secure management of multiple tenants further enhance data security and compliance.

NetApp ONTAP also features SnapLock, which provides regulatory-compliant data retention with high levels of integrity, performance, and retention at a low total cost of ownership. It is fully integrated with NetApp ONTAP® 9 and offers protection against malicious acts, rogue administrators, and ransomware.

The solution encompasses NSE/NVE encryption for in-flight and data-at-rest encryption, multifactor admin access, and multi-admin verification. Active IQ provides AI-informed predictive analytics and corrective action, while QoS ensures quality of service workload control. The management and automation integration is intuitive through SysMgr/GUI/CLI/API. FabricPool enables automatic data tiering, and the solution offers efficiency through inline data compression, deduplication, and compaction. NetApp guarantees meeting workload efficiency goals at no cost to the customer.

NetApp ONTAP supports various protocols, including NVMe/FC, FC, NVMe/TCP, iSCSI, NFS, SMB, and S3, making it a unified storage solution. Overall, NetApp ONTAP provides extensive enterprise features, robust security, compliance, efficiency, and versatility to meet diverse storage needs.

Dremio overview

Dremio is the Unified Lakehouse Platform for self-service analytics and AI. The Dremio Unified Analytics Platform brings users closer to the data with lakehouse flexibility, scalability, and performance at a fraction of the cost of legacy data warehouse solutions. Dremio enables "shift-left" analytics to eliminate complex and costly data integration and ETL, delivering seamless enterprise-scale analytics with no data movement. Dremio also features:

  • Easy-to-use self-service analytics enabled through a universal semantic layer and a tightly integrated, highly performant SQL query engine, making it easier to connect, govern, and analyze all data, both in the cloud and on-premises.

  • Dremio’s Apache Iceberg-native lakehouse management capabilities simplify data discovery, and automate data optimization, delivering high-performance analytics with Git-inspired data versioning.

  • Foundationally built on open source and open standards, Dremio enables companies avoid lock-in and remain positioned for innovation. Enterprise companies trust Dremio as the easiest-to-use lakehouse platform with the best price performance across all workloads.

What value does the Dremio and NetApp Hybrid Iceberg Lakehouse solution deliver to customers?

  • Improved Data Management and Accessibility: Dremio is well-known for its data lakehouse platform that enables organizations to query data directly from their data lakes at high speed. NetApp, on the other hand, is a leading provider of cloud data services and data storage solutions. The joint offer provides customers with a comprehensive solution for storing, managing, accessing, and analyzing their enterprise’s data efficiently and efficiently.

  • Performance Optimization: With NetApp's expertise in data storage and Dremio's capabilities in data processing and data optimization, the partnership offers a solution that improves the performance of data operations, reduces latency, and increases speed to business insight. Dremio has even delivered performance benefits to NetApp’s own internal IT analytical infrastructure.

  • Scalability: Both Dremio and NetApp offer a solution that is designed to scale. The joint solution provides customers with highly-scalable data storage, data management, and analytics environments. In a Hybrid Iceberg Lakehouse environment, the Dremio SQL query engine paired with NetApp StorageGRID delivers unparalleled scalability, concurrency, and query performance, capable of handling the analytical needs of any business.

  • Data Security and Governance: Both companies have a strong focus on data security and governance. Together, they offer robust security and data governance features, ensuring that data is protected and that data governance requirements are met. Features such as role-based and fine-grain access controls, comprehensive auditing, end-to-end data lineage, unified identity management, and SSO with an extensive compliance and security framework ensure companies' analytical data environments are secure and governed.

  • Cost Efficiency: By integrating Dremio's data lake engine with NetApp's storage solutions, customers can reduce costs associated with data management and data movement. Organizations are also able to move from legacy data lake environments to a more modern lakehouse solution composed of NetApp and Dremio. This Hybrid Iceberg Lakehouse solution delivers high-speed query performance and market-leading query concurrency that lowers TCO and reduces time to business insight.