Use case 4: Data protection and multicloud connectivity
Suggest changes
PDFs
This use case is relevant for a cloud service partner tasked with providing multicloud connectivity for customers' big data analytics data.
Scenario
In this scenario, IoT data received in AWS from different sources is stored in a central location in NPS. The NPS storage is connected to Spark/Hadoop clusters located in AWS and Azure enabling big data analytics applications running in multiple clouds accessing the same data.
Requirements and challenges
The main requirements and challenges for this use case include:
-
Customers want to run analytics jobs on the same data using multiple clouds.
-
Data must be received from different sources such as on-premises and cloud through different sensors and hubs.
-
The solution must be efficient and cost-effective.
-
The main challenge is to build a cost-effective and efficient solution that delivers hybrid analytics services between on-premises and across different clouds.
Solution
This image illustrates the data protection and multicloud connectivity solution.
As shown in the figure above, data from sensors is streamed and ingested into the AWS Spark cluster through Kafka. The data is stored in an NFS share residing in NPS, which is located outside of the cloud provider within an Equinix data center. Because NetApp NPS is connected to Amazon AWS and Microsoft Azure through Direct Connect and Express Route connections, respectively, customers can access the NFS data from both Amazon and AWS analytics clusters. This approach solves having cloud analytics across multiple hyperscalers.
Consequently, because both on-premises and NPS storage runs ONTAP software, SnapMirror can mirror the NPS data into the on-premises cluster, providing hybrid cloud analytics across on-premises and multiple clouds.
For the best performance, NetApp typically recommends using multiple network interfaces and direct connection/express routes to access the data from cloud instances.