Combining Pattern Classifiers: Methods and Algorithms, 2nd Edition

2 BASE CLASSIFIERS

The classifiers whose decisions are combined to form the ensemble are called “base classifiers.” This chapter details some of the most popular base classifier models.

2.1 LINEAR AND QUADRATIC CLASSIFIERS

2.1.1 Linear Discriminant Classifier

Linear and quadratic classifiers are named after the type of discriminant functions they use. Let be the object to classify in one of c classes. Let be a vector with coefficients and w_i0 be a constant free term. A linear classifier is any set of c linear functions, one for each class, , i = 1, …, c,

(2.1)

The tag of the largest g_i(x) determines the class label.

2.1.1.1 Training Linear Discriminant Classifier.

A straightforward way to train a linear discriminant classifier (LDC) is shown in Figure 2.2 and detailed below:

1. Estimate the prior probabilities for the classes. Let N_i be the number of objects in the data set Z from class ω_i, i = 1, …c, and y_j ∈ Ω be the class label of z_j ∈ Z. Then
   (2.2)
2. Calculate estimates of ...