Managing TaskTracker

TaskTrackers are MapReduce daemon processes that run on slave nodes. They accept tasks assigned by the JobTracker on the master node and fork JVM processes/threads to run the tasks. TaskTracker is also responsible for reporting the progress of the tasks as well as its health status using heartbeat.

Hadoop maintains three lists for TaskTrackers: blacklist , gray list, and excluded list. TaskTracker black listing is a function that can blacklist a TaskTracker if it is in an unstable state or its performance has been downgraded. For example, when the ratio of failed tasks for a specific job has reached a certain threshold, the TaskTracker will be blacklisted for this job. Similarly, Hadoop maintains a gray list of nodes by identifying ...

Get Hadoop Operations and Cluster Management Cookbook now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.