Python Machine Learning By Example - Second Edition

Best practice 20 – monitoring model performance

The machine learning system is now up and running. To make sure everything is on the right track, we need to conduct a performance check on a regular basis. To do so, besides making a prediction in real time, we should record the ground truth at the same time.

Continue the previous diabetes example with a performance check as follows:

>>> from sklearn.metrics import r2_score>>> print('Health check on the model, R^2:                  {0:.3f}'.format(r2_score(y_new, predictions)))Health check on the model, R^2: 0.613

We should log the performance and set an alert for a decayed performance.