Practical Predictive Analytics

Goodness of fit – Hosmer-Lemeshow test

This goodness of fit test will look at subgroups of the predicted probabilities, for example, the probability ranges grouped into 10 (deciles), and measuring whether or not the prediction of the response variable is significantly different from what would be expected by chance.

First the data is group by their predicted probabilities from lowest to highest, and then binned into equally-sized groups.

Then a chi-square test is used to compare the predicted probability of each binned group to their expected probabilities. In a good model, predictions should do better than what is expected for most of the binned groups. If it does not, the groups which are problematic will be indicated in the predicted vs. ...