Learning YARN

Chapter 7. Writing Your Own YARN Applications

In the first chapter, we talked about the shortcomings of Hadoop 1.x framework. Hadoop 1.x framework was restricted to MapReduce programming only. You had to write data processing logic as map and reduce tasks. With the introduction of YARN in Hadoop 2.x version, you can now execute different data processing algorithms over the data stored in HDFS. YARN separates the resource management and the data processing frameworks into two different components, ResourceManager and ApplicationMaster.

In the last few chapters, you learned about the application execution flow, and how YARN components communicate and manage the life cycle of an application. You executed a MapReduce application over a YARN cluster ...