Skip to main content
NetApp Solutions

Solution

Contributors banum-netapp
PDFs

Confluent and NetApp AFF A900 storage controller powered by ONTAP are distributed systems designed for data streams. Both are horizontally scalable, fault tolerant, and provide excellent performance under load. They complement each other in distributed data streaming and stream processing with lower storage costs with data reduction technologies that minimize the data footprint. The AFF A900 storage controller provides great performance, while allowing the decoupling of compute and data storage resources. This simplifies system administration and allows resources to be scaled independently.

Image depicting the solution overview.

Solution architecture details

This section covers the hardware and software used for performance verification in Confluent Platform deployment with NetApp ONTAP for tiered storage. The following table covers the solution architecture and base components.

Platform component Environment configuration

Confluent Platform version 6.2

  • 3 x zookeepers

  • 8 x broker servers

  • 5 x tools servers

  • 1 x Grafana

  • 1 x control center

Operating system on all nodes

Linux (ubuntu 18.04)

NetApp ONTAP for warm buckets

  • 1 x AFF A900 high-availability (HA) pair

  • 4 x 24 x 800 SSDs

  • S3 protocol

  • 100GbE

15 Fujitsu PRIMERGY RX2540 servers

  • 2 CPUs; 16 physical cores total

  • Intel Xeon

  • 256GB physical memory

  • 100GbE dual port