Machine Learning with scikit-learn Quick Start Guide

Calibration plot

A calibration plot, as the name suggests, tells you how well calibrated your model is. A well-calibrated model will have a prediction score equal to the fraction of the positive class (in this case, the fraudulent transactions). In order to plot a calibration plot, we use the following code:

#Extracting the probabilites that the positive class will be predictedknn_proba = knn_classifier.predict_proba(X_test)log_proba = logistic_regression.predict_proba(X_test)#Storing probabilities in a listprobas = [knn_proba, log_proba]# Storing the model names in a list model_names = ["k_nn", "Logistic Regression"]#Creating the calibration plotskplt.metrics.plot_calibration_curve(y_test, probas, model_names)plt.show()

This results ...