Apache Spark Quick Start Guide

MLlib

MLlib is one of the machine learning libraries of Spark. MLlib provides a set of data types that uses RDD to represent data points. Some of the common datatypes are vectors and labeled points. In MLlib, the supervised algorithm uses RDDs of labeled points and unsupervised algorithm uses vectors to train a model. The MLlib API provides a wide range of machine learning algorithms. Unfortunately, discussing all of them is outside the scope of this book. But, we'll implement one of the supervised learning algorithms and walk you through some of the basic concepts.

One of the most famous supervised learning algorithms is linear regression. Before we create our model, let's look at the input data. To understand the algorithm, I have created ...