3 Designing a Kafka project

This chapters covers

  • Designing a real-world Kafka project
  • Determining which data format to use
  • Existing issues impacting data usage
  • Deciding when data transformation takes place
  • How Kafka Connect helps us start a data-streaming path

In our previous chapter, we saw how we can work with Kafka from the command line and how to use a Java client. Now, we will expand on those first concepts and look at designing various solutions with Kafka. We will discuss some questions to consider as we lay out a strategy for the example project we’ll start in this chapter. As we begin to develop our solutions, keep in mind that, like most projects, we might make minor changes along the way and are just looking for a place to jump in ...

Get Kafka in Action now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.