Big Data Simplified

Working with Big Data inPython | 253

PHP, Scala, Perl, UNIX and many more. This utility allows us to create and run Map/Reduce jobs

with any executable or script as the mapper and/or the reducer. A sample of the Hadoop execu-

tion is given below.

$HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/tools/lib/hadoop-

streaming.jar \

-input myInputDirs \

-output myOutputDir \

-mapper /bin/cat \

-reducer /bin/wc

9.4.2 Python MapReduce Code

mapper.py

#!/usr/bin/python

import sys

#Word Count Example

# input comes from standard input STDIN

for line in sys.stdin:

line = line.strip() #remove leading and trailing whitespaces

words = line.split() ...