April 2018
Intermediate to advanced
260 pages
6h 31m
English
While the mapper's job is to extract and emit some form of data to be aggregated, the reducer's job is to perform that aggregation. In this example, the reducer receives the full list of (word, 1) pairs and just adds up the counts (1, in this case) for each word. If we were to perform a different aggregation, the reducer function would need to change.
Rather than counting the number of occurrences, let's calculate the average length of a word. In this case, both our mapper and our reducer will need updating with the more significant changes happening within the reducer. The following code block changes our example to calculate the average word length for a body of text broken up into words:
def mapper(word): return len(word) ...
Read now
Unlock full access