R Statistics Cookbook

How it works...

After we compute the predictions for our model, we can pass them, along with the real labels, to any proper function that computes the ROC and PR curves. These are obtained by changing the threshold and recomputing the TPR, FPR, recall, and precision.

ROC curves have two main advantages: they are robust to class imbalance (accuracy is not robust in any way), and they allow us to compare models irrepective of the threshold. They allow us to compare models along all possible thresholds. This is much better than for example the F1 score, or Cohen's Kappa because they need us to fix a threshold (they are usually calculated assuming a threshold of 0.5). It is worth noting that classification models can work well for certain thresholds ...