Minimum and maximum

The minimum and maximum calculation for a specific field is a commonly used use case in MapReduce. Once the mapper completes its operation, the reducer simply iterates through all the key values and finds out the minimum and maximum in the key grouping: 

  • Writables: The idea behind writing custom writables was to save extra effort in splitting data at the reducer side and avoiding unnecessary problems that can occur from the delimiter. Most of the time, we choose the delimiter that is already present in the record and then it leads to the incorrect mapping of records with the field. 

We will use the following import packages:

import org.apache.Hadoop.io.IntWritable;import org.apache.Hadoop.io.LongWritable;import org.apache.Hadoop.io.Text; ...

Get Mastering Hadoop 3 now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.