Machine Learning Pocket Reference

Chapter 12. Metrics and Classification Evaluation

We’ll cover the following metrics and evaluation tools in this chapter: confusion matrices, various metrics, a classification report, and some visualizations.

This will be evaluated as a decision tree model that predicts Titanic survival.

Confusion Matrix

A confusion matrix can aid in understanding how a classifier performs.

A binary classifier can have four classification results: true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN). The first two are correct classifications.

Here is a common example for remembering the other results. Assuming positive means pregnant and negative is not pregnant, a false positive is like claiming a man is pregnant. A false negative is claiming that a pregnant woman is not (when she is clearly showing) (see Figure 12-1). These last two types of errors are referred to as type 1 and type 2 errors, respectively (see Table 12-1).

Another way to remember these is that P (for false positive) has one straight line in it (type 1 error), and N (for false negative) has two vertical lines in it.

Figure 12-1. Classification errors.

Table 12-1. Binary classification results from a confusion matrix

Actual	Predicted negative	Predicted positive
Actual negative	True negative	False positive (type 1)
Actual positive	False negative (type 2)	True positive

Here is the ...