Learning Data Mining with Python - Second Edition

A word count example

Any actual implementation of MapReduce is a little more complex than just using a map and reduce step. Both steps are invoked using keys, which allows for the separation of data and tracking of values.

The map function takes a key-value pair and returns a list of key/value pairs. The keys for the input and output don't necessarily relate to each other.

For example, for a MapReduce program that performs a word count, the input key might be a sample document's ID value, while the output key would be a given word. The input value would be the text of the document and the output value would be the frequency of each word.  We split the document to get the words, and then yield each of the word, count pairs. The word here is ...