Machine Learning Algorithms - Second Edition

Discriminant analysis

Let's suppose we consider a multi-class classification problem where the conditional probability for a sample x_i ∈ ℜ^m to belong to the y_j class can be modeled as a multivariate Gaussian distribution (X is assumed to be made up of independent and identically distributed (i.i.d) variables with extremely low collinearities):

In this case, the class j is fully determined by the mean vector μ_j and the covariance matrix Σ_j. If we apply the Bayes' theorem, we can obtain the posterior probability p(y_j|x_i):

Considering the discussion ...