Machine Learning with scikit-learn Quick Start Guide

The confusion matrix

Until now, we have used the accuracy as the sole measure of model performance. That was fine, because we have a balanced dataset. A balanced dataset is a dataset in which there are almost the same numbers of labels for each category. In the dataset that we are working with, 8,000 labels belong to the fraudulent transactions, while 12,000 belong to the non-fraudulent transactions.

Imagine a situation in which 90% of our data had non-fraudulent transactions, while only 10% of the transactions had fraudulent cases. If the classifier reported an accuracy of 90%, it wouldn't make sense, because most of the data that it has seen thus far were the non-fraudulent cases and it has seen very little of the fraudulent cases. So, ...