O'Reilly logo

Apache Mahout Essentials by Jayani Withanawasam

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

From Hadoop MapReduce to Spark

Let's take a look at the journey from MapReduce to Spark.

Problems with Hadoop MapReduce

Even though MapReduce provides a suitable programming model for batch data processing, it does not perform well with real-time data processing. When it comes to iterative machine learning algorithms, it is necessary to carry information across iterations. Moreover, an intermediate outcome needs to be persisted during each iteration. Therefore, it is necessary to store and retrieve temporary data from the Hadoop Distributed File System (HDFS) very frequently, which incurs significant performance degradation.

Machine learning algorithms that can be written in a certain form of summation (algorithms that fit in the statistical query ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required