tabulating the number of errors. If the test set is the same as the training
set, the performance estimates will be optimistically biased. If it includes none of
the training data, they will be pessimistically biased. If the test set is large, the
effects of this bias will be slight. If the number of available preclassified objects is
small, one can use the ‘‘round robin’’ or ‘‘leave one out’’ method. Here the classifier
is trained on all but one of the objects and tested on the remaining object. This
process is repeated until every object has been used for testing. The results of
the various experiments are then averaged together to estimate the error rates.
11.5.1 The Confusion Matrix
A very handy tool for specifying the accuracy of a multiclass