Practical Predictive Analytics

Regularization Models

Regularization models are an alternative to spark.glm; you can also run spark.logit and supply regularization parmeters to the independent variables.

By default, spark.logit will yield the same results as spark.glm without regularization parameters:

model2 <- spark.logit(df, outcome ~ pregnant + glucose + pressure + triceps + insulin + pedigree + age)

To verify this, run both spark.logit and spark.glm and verify that the results are identical.

Once you have verified this, you may add regularization parameters if you wish to smooth out the model by flattening the coefficients or set some of them to 0.

The model which has been run below only includes glucose, insulin, and pressure as predictors. Since the elasticnetparm ...