Chapter 3. Transporting the data from collection tier: decoupling the data pipeline

This chapter covers

  • Understanding the need for the message queuing tier
  • Understanding message durability
  • Accommodating offline consumers
  • Understanding message delivery semantics
  • Choosing the right technology

So far I’ve talked about the role of handling the incoming data, not the output of data, from the collection tier. This chapter focuses on transporting data from the collection tier to the rest of the streaming pipeline. Although I may mention the collection and analysis tiers in the discussion, the discussion will only be concerned with getting messages from or to those tiers via the message queuing tier. Figure 3.1 shows our streaming architecture ...

Get Streaming Data now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.