Mastering Apache Pulsar

Chapter 8. Pulsar Functions

In Chapter 7 I hinted that Pulsar Functions are the bedrock technology used in Pulsar IO. Now we will explore what makes Pulsar Functions unique and why they contribute to make Pulsar an enticing option for an event streaming system.

I’ve worked with several stream processing engines during my career, from Apache Storm and Spark Streaming to Flink and Kafka Streams. Pulsar Functions are a unique and, perhaps, one of the more agile approaches to the stream processing problem (see Figure 8-1). Pulsar Functions take a problem that requires a large and cumbersome runtime and distill it down to a problem that only requires creating functions that have a topic as an input and a topic as an output. In this chapter you’ll learn the reasoning behind Pulsar Functions as a stream processing system. You’ll also learn what makes Pulsar Functions unique and what its limitations are. Finally, we’ll talk about the deployment models for Pulsar Functions and walk through some use cases.

Figure 8-1. Pulsar Functions have a topic as input and a topic as output, and they perform some logic on the data they ingest.

Stream Processing

Organizing and collecting event streams is the first requirement for getting value from real-time data, but enriching, routing, deleting, and triggering actions based on event streams is where the rubber meets the road. Stream processing is ...