Mastering Machine Learning with scikit-learn - Second Edition

Generative and discriminative models

In classification tasks, our goal is to learn the parameters of a model that optimally maps features of the explanatory variables to the response variable. All of the classifiers that we have previously discussed are discriminative models, which learn a decision boundary that is used to discriminate between classes. Probabilistic discriminative models, such as logistic regression, learn to estimate the conditional probability P(y|x); they learn to estimate which class is most likely given the input features. Non-probabilistic discriminative models, such as KNN, directly map features to classes.

Generative models do not directly learn a decision boundary. Instead, they model the joint probability distribution ...