Hadoop: Data Processing and Modelling

Time for action – causing task failure

Let's cause a task to fail; before we do, we will need to modify the default timeouts:

1. Add this configuration property to mapred-site.xml:

   <property>
   <name>mapred.task.timeout</name>
   <value>30000</value>
   </property>

2. We will now modify our old friend WordCount from Chapter 3, Understanding MapReduce. Copy WordCount3.java to a new file called WordCountTimeout.java and add the following imports:

   import java.util.concurrent.TimeUnit ;
   import org.apache.hadoop.fs.FileSystem ;
   import org.apache.hadoop.fs.FSDataOutputStream ;

3. Replace the map method with the following one:

    public void map(Object key, Text value, Context context ) throws IOException, InterruptedException { String lockfile = "/user/hadoop/hdfs.lock" ; Configuration ...