The preceding metrics are good for binary classifiers which produce a hard output; they only tell whether the class should be assigned a positive label or negative. If, instead, our model outputs some score such that the higher the values of the score the more likely the item is to be positive, then the binary classifier is called a ranking classifier.
Most of the models can output probabilities of belonging to a certain class, and we can use it to rank examples such that the positives are likely to come first.
The ROC curve visually tells us how good a ranking classifier separates positive examples from negative ones. The way a ROC curve is built is as follows:
- Sort the observations by their score and then start from ...
