Combining Pattern Classifiers: Methods and Algorithms, 2nd Edition

4 COMBINING LABEL OUTPUTS

How do we combine the outputs of the individual classifiers in the ensemble? Numerous theoretical analyses [145, 207, 235, 241, 249, 260, 394], experimental comparisons [109, 152, 222, 382, 434, 436], and reviews [392, 439] look for the answer to this question.

4.1 TYPES OF CLASSIFIER OUTPUTS

Consider a classifier ensemble consisting of L classifiers in the set and a set of classes Ω = {ω₁, …, ω_c}. Xu et al. [425] distinguish between three types of classifier outputs:

• Class labels. (The abstract level.) Each classifier D_i produces a class label s_i ∈ Ω, i = 1, …, L. Thus, for any object to be classified, the L classifier outputs define a vector s = [s₁, …, s_L]^T ∈ Ω^L. At the abstract level, there is no information about the certainty of the guessed labels, nor are any alternative labels suggested. By definition, any classifier is capable of producing a label for x, so the abstract level is universal.
• Ranked class labels. The output of each D_i is a subset of the class labels Ω, ranked in order of plausibility [184, 391]. This type is especially suitable for problems with a large number of classes, such as character, face, and speaker recognition.
• Numerical support for the classes. (The measurement level.) Each classifier D_i produces a c-dimensional vector ...