Python Machine Learning By Example - Second Edition

The mechanics of Naïve Bayes

Let's start with understanding the magic behind the algorithm—how Naïve Bayes works. Given a data sample x with n features, x₁, x₂, …, x_n (x represents a feature vector and x = (x₁, x₂, …, x_n)), the goal of Naïve Bayes is to determine the probabilities that this sample belongs to each of K possible classes y₁, y₂, …, y_K, that is P( y_k |x) or P(x₁, x₂, …, x_n), where k = 1, 2, …, K. It looks no different from what we have just dealt with: x, or x₁, x₂, …, x_n, is a joint event that the sample has features with values x₁, x₂, …, x_n respectively, y_k is an event that the sample belongs to class k. We can apply Bayes' theorem right away:

Let's look at each component in detail:

• P (y_k) portrays how classes are distributed, ...