Mastering Machine Learning with Spark 2.x

Spark Naive Bayes model

Next, let's look to employ Spark's implementation of Naive Bayes. As a reminder, we purposely stay away from going into the algorithm itself as this has been covered in many machine learning books; instead, we will focus on the parameters of the model and ultimately, how we can "deploy" these models in a Spark streaming application later on in this chapter.

Spark's implementation of Naive Bayes is relatively straightforward, with just a few parameters we need to keep in mind. They are mainly as follows:

• getLambda: Sometimes referred to as "additive smoothing" or "laplace smoothing," this parameter allows us to smooth out the observed proportions of our categorical variables to create a more uniform distribution. This ...