Skip to Main Content
Exercises in Programming Style
book

Exercises in Programming Style

by Cristina Videira Lopes
November 2015
Intermediate to advanced content levelIntermediate to advanced
304 pages
5h 23m
English
Chapman and Hall/CRC
Content preview from Exercises in Programming Style

Chapter 30

Map Reduce

image

30.1 Constraints

  • Input data is divided in blocks.
  • A map function applies a given worker function to each block of data, potentially in parallel.
  • A reduce function takes the results of the many worker functions and recombines them into a coherent output.

30.2 A Program in this Style

  1 #!/usr/bin/env python
  2 import sys, re, operator, string
  3
  4 #
  5 # Functions for map reduce
  6 #
  7 def partition(data_str, nlines):
  8 """
  9 Partitions the input data_str (a big string)
 10 into chunks of nlines.
 11 """
 12 lines = data_str.split('\n')
 13 for i in xrange(0, len(lines), nlines):
 14  yield '\n'.join(lines[i:i+nlines])
 15
 16 def ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Exercises for Programmers

Exercises for Programmers

Brian P. Hogan
Street Coder

Street Coder

Sedat Kapanoglu
Handbook of Constraint Programming

Handbook of Constraint Programming

Francesca Rossi, Peter van Beek, Toby Walsh

Publisher Resources

ISBN: 9781482227376