Concurrent Data Processing in Elixir

Performing Reduce Concurrently with Partitions

In addition to map and filter, the reduce function in the Enum module is another frequently used tool when working with data. It is also the backbone of other functions in this module, including group_by, uniq_by, and take_sort. In this section, you will see how to use Flow.reduce/3 to count the number of working airports in each country present in the dataset in parallel. Then we’ll move on to other convenience functions for grouping and sorting data. But before we begin, it is useful to understand how reduce works in Flow.

In reduce we have something called an accumulator, which could be any data type, and holds the latest state. Each element in the list we want to process is given to a reducer ...