Mastering Machine Learning with Spark 2.x

Using models for scoring

In the previous sections, we explored different data processing steps, and built and evaluated several models to predict the loan status and interest rates for the accepted loans. Now, it is time to use all built artifacts and compose them together to score new loans.

There are multiple steps that we need to consider:

1. Data cleanup
2. The emp_title column preparation pipeline
3. The desc column transformation into a vector representing significant words
4. The binomial model to predict loan acceptance status
5. The regression model to predict loan interest rate

To reuse these steps, we need to connect them into a single function that accepts input data and produces predictions involving loan acceptance status and interest rate. ...