Mastering Kafka Streams and ksqlDB

Chapter 8. Getting Started with ksqlDB

The story of ksqlDB is one of simplification and evolution. It was built with the same goal as Kafka Streams: simplify the process of building stream processing applications. However, as ksqlDB evolved, it became clear that its goals were much more ambitious than even Kafka Streams. That’s because it not only simplifies how we build stream processing applications, but also how we integrate these applications with other systems (including those external to Kafka). It does all of this with a SQL interface, making it easy for beginners and experts alike to leverage the power of Kafka.

Now, I know what you’re thinking: why would you need to know both Kafka Streams and ksqlDB, and can you rip out one of the sections of this book and sell it on Craigslist to recover some of your investment? Actually, both Kafka Streams and ksqlDB are excellent tools to have in your stream processing toolbelt, and complement each other quite well. You can use ksqlDB for stream processing applications that can be expressed in SQL, and for easily setting up data sources and sinks to create end-to-end data processing pipelines using a single tool. On the other hand, you can use Kafka Streams for more complex applications, and your knowledge of that library will only deepen your understanding of ksqlDB since it’s actually built on top of Kafka Streams.

I was going to save the part about ksqlDB being built on top of Kafka Streams as a big reveal later in this chapter, ...