Machine Learning in Java - Second Edition

Evaluation and prediction error metrics

We built a model, but we do not know if it can be trusted. To estimate its performance, we can apply a cross-validation technique that was explained in Chapter 1, Applied Machine Learning Quick Start.

Weka offers an Evaluation class for implementing cross-validation. We pass the model, data, number of folds, and an initial random seed, as follows:

Classifier cl = new J48(); 
Evaluation eval_roc = new Evaluation(data); 
eval_roc.crossValidateModel(cl, data, 10, new Random(1), new Object[] {}); 
System.out.println(eval_roc.toSummaryString()); 

The evaluation results are stored in the Evaluation object.

A mix of the most common metrics can be invoked by calling the toString() method. Note that the output ...