Conclusion

09/10/2024 Contributors

This document provides best practice guidelines for using Confluent Tiered Storage with NetApp storage, including verification tests, tiered storage performance results, tuning, Confluent S3 connectors, and the self-balancing feature. Considering ILM policies, Confluent performance with multiple performance tests for verification, and industry-standard S3 APIs, NetApp StorageGRID object storage is an optimal choice for Confluent tiered storage.

Where to find additional information

To learn more about the information that is described in this document, review the following documents and/or websites:

What is Apache Kafka

https://www.confluent.io/what-is-apache-kafka/
NetApp Product Documentation

https://www.netapp.com/support-and-training/documentation/
S3-sink parameter details

https://docs.confluent.io/kafka-connect-s3-sink/current/configuration_options.html#s3-configuration-options
Apache Kafka

https://en.wikipedia.org/wiki/Apache_Kafka
Infinite Storage in Confluent Platform

https://www.confluent.io/blog/infinite-kafka-storage-in-confluent-platform/
Confluent Tiered Storage - Best practices and sizing

https://docs.confluent.io/platform/current/kafka/tiered-storage.html#best-practices-and-recommendations
Amazon S3 sink connector for Confluent Platform

https://docs.confluent.io/kafka-connect-s3-sink/current/overview.html
Kafka sizing

https://eventsizer.io
StorageGRID sizing

https://fusion.netapp.com/
Kafka use cases

https://kafka.apache.org/uses
Self-balancing Kafka clusters in confluent platform 6.0

https://www.confluent.io/blog/self-balancing-kafka-clusters-in-confluent-platform-6-0/

https://www.confluent.io/blog/confluent-platform-6-0-delivers-the-most-powerful-event-streaming-platform-to-date/

Conclusion

Creating your file...

Where to find additional information