O'Reilly logo

Hadoop Operations and Cluster Management Cookbook by Shumin Guo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Managing TaskTracker

TaskTrackers are MapReduce daemon processes that run on slave nodes. They accept tasks assigned by the JobTracker on the master node and fork JVM processes/threads to run the tasks. TaskTracker is also responsible for reporting the progress of the tasks as well as its health status using heartbeat.

Hadoop maintains three lists for TaskTrackers: blacklist , gray list, and excluded list. TaskTracker black listing is a function that can blacklist a TaskTracker if it is in an unstable state or its performance has been downgraded. For example, when the ratio of failed tasks for a specific job has reached a certain threshold, the TaskTracker will be blacklisted for this job. Similarly, Hadoop maintains a gray list of nodes by identifying ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required