Mastering Machine Learning with Spark 2.x

Super-learner model

Now, we will combine the prediction power for all of these algorithms to generate a "super-learner" with the help of a neural network, which takes each model's prediction as input and then, tries to come up with a better prediction, given the guesses of the individually trained models. At a high level, the architecture would look something like this:

We will explain further the intuition behind building a "super-learner" and the benefits of this approach, and teach you how to build your Spark streaming application, which will take in your text (that is, a movie review that you will write) and run it through the prediction ...