Advanced Machine Learning with R

Model comparison

A useful tool for a classification model comparison is the Receiver Operating Characteristic (ROC) chart. ROC is a technique for visualizing, organizing, and selecting classifiers based on their performance (Fawcett, 2006). On the ROC chart, the y axis is the True Positive Rate (TPR), and the x axis is the False Positive Rate (FPR).

To create a ROC chart in R, you can use the ROCR package. I think this is a great package and allows you to build a chart in just three lines of code. The package also has an excellent companion website (with examples and a presentation) that can be found at the following link: http://rocr.bioinf.mpi-sb.mpg.de/.

For each model, you create a prediction object of the actual labels and the predicted ...