Python Data Science Handbook, 2nd Edition

Chapter 41. In Depth: Naive Bayes Classification

The previous four chapters have given a general overview of the concepts of machine learning. In the rest of Part V, we will be taking a closer look first at four algorithms for supervised learning, and then at four algorithms for unsupervised learning. We start here with our first supervised method, naive Bayes classification.

Naive Bayes models are a group of extremely fast and simple classification algorithms that are often suitable for very high-dimensional datasets. Because they are so fast and have so few tunable parameters, they end up being useful as a quick-and-dirty baseline for a classification problem. This chapter will provide an intuitive explanation of how naive Bayes classifiers work, followed by a few examples of them in action on some datasets.

Bayesian Classification

Naive Bayes classifiers are built on Bayesian classification methods. These rely on Bayes’s theorem, which is an equation describing the relationship of conditional probabilities of statistical quantities. In Bayesian classification, we’re interested in finding the probability of a label $L$ given some observed features, which we can write as $P\left(L\right|\mathrm{features})$. Bayes’s theorem tells us how to express this in terms of quantities we can compute more directly:

$$P\left(L\right|\mathrm{features})=\frac{P\left(\mathrm{features}\right|L\left)P\right(L)}{P\left(\mathrm{features}\right)}$$

If we are trying to decide between two labels—let’s call them $L_1$ and $L_2$—then one way to make this decision is ...