Big Data Analytics Beyond Hadoop: Real-Time Applications with Storm, Spark, and More Hadoop Alternatives

6. Conclusions: Big Data Analytics Beyond Hadoop Map-Reduce

With the advent of Hadoop 2.0—the new release of Hadoop known as Yet Another Resource Negotiator (YARN)—the beyond–Map-Reduce (MR) thinking has been solidified. As is explained in this chapter, Hadoop YARN separates the resource scheduling part from the MR paradigm. It should be noted that in Hadoop 1.0, the first-generation Hadoop, the scheduling was tied with the MR paradigm—implying that the only processing that was possible on Hadoop Distributed File System (HDFS) data was the MR type or its orchestrations. This has been addressed in YARN, which enables HDFS data to be processed by any non-MR paradigm as well. The implication is an acknowledgment of the fact that MR is not the only ...