Chapter 6. YARN – Bringing Other Paradigms to Hadoop
Yet Another Resource Negotiator (YARN) is a cluster resource management layer that was introduced in Hadoop 2.0. As we saw briefly in Chapter 1, Hadoop 2.X, YARN separates out the responsibilities of the JobTracker daemon. JobTracker was responsible for:
- Resource arbitration within a Hadoop cluster
- MapReduce job management
The problem with the JobTracker model was that it became the single point of failure in the compute layer of a Hadoop cluster. Any failure in JobTracker meant trashing the running jobs and starting all over again. JobTracker's singular nature also became a scaling bottleneck. All job communications, scheduling, and resource management were controlled by the JobTracker master ...
Get Mastering Hadoop now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
