Concurrent Data Processing in Elixir

Introducing GenStage

GenStage was originally developed by José Valim, the creator of Elixir, and released in July 2016. As he described it in the official announcement:

“GenStage is a new Elixir behaviour for exchanging events with back-pressure between Elixir processes.”

In the previous chapter, we used the GenServer behaviour to build long-running server processes. The GenStage behaviour, as its name suggests, is used to build stages. Stages are also Elixir processes and they’re our building blocks for creating data-processing pipelines.

Stages are simple but very powerful. They can receive events and use them to do some useful work. They can also send events to the next stage in the pipeline. You can do that by connecting stages to each ...