Pattern Recognition, 4th Edition

5.10. A Hint on Generalization Theory

So far in this book, two major issues have occupied us: the design of the classifier and its generalization capabilities. The design of the classifier involved two stages: the choice of the classifier type and the choice of the optimality criterion. The generalization capabilities led us to seek ways to reduce the feature space dimensionality. In this section we will point out some important theoretical results that relate the size N of the training data set and the generalization performance of the designed classifier.

To this end, let us summarize a few necessary basic steps and definitions.

• Let F be the set of all the functions f that can be realized by the adopted classifier scheme. For example, ...