Classification, Parameter Estimation and State Estimation, 2nd Edition

6Supervised Learning

One method for the development of a classifier or an estimator is the so-called model-based approach. Here, the required availability of the conditional probability densities and the prior probabilities are obtained by means of general knowledge of the physical process and the sensory system in terms of mathematical models. The development of the estimators for the backscattering coefficient, discussed in Chapter 4, follows such an approach.

In many other applications, modelling the process is very difficult, if not impossible. For instance, in the mechanical parts application, discussed in Chapter 3, the visual appearance of the objects depends on many factors that are difficult to model. The alternative to the model-based approach is the learning from examples paradigm. Here, it is assumed that in a given application a population of objects is available. From this population, some objects are selected. These selected objects are called the samples. Each sample is presented to the sensory system, which returns the measurement vector associated with that sample. The purpose of learning (or training) is to use these measurement vectors of the samples to build a classifier or an estimator.

The problem of learning has two versions: supervised and unsupervised, that is with or without knowing the true class/parameter of the sample (see Figure 6.1). This chapter addresses the first version. Chapter 8 deals with unsupervised learning.

Figure 6.1 Training sets. ...