Time for action – causing task failure

Let's cause a task to fail; before we do, we will need to modify the default timeouts:

  1. Add this configuration property to mapred-site.xml:
    <property>
    <name>mapred.task.timeout</name>
    <value>30000</value>
    </property>
  2. We will now modify our old friend WordCount from Chapter 3, Understanding MapReduce. Copy WordCount3.java to a new file called WordCountTimeout.java and add the following imports:
    import java.util.concurrent.TimeUnit ;
    import org.apache.hadoop.fs.FileSystem ;
    import org.apache.hadoop.fs.FSDataOutputStream ;
  3. Replace the map method with the following one:
     public void map(Object key, Text value, Context context ) throws IOException, InterruptedException { String lockfile = "/user/hadoop/hdfs.lock" ; Configuration ...

Get Hadoop: Data Processing and Modelling now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.