Data Science Using Python and R

Chapter 8NAÏVE BAYES CLASSIFICATION

8.1 INTRODUCTION TO NAÏVE BAYES

Of course, classification modeling is not restricted to decision trees. Many other classification methods are available, including Naïve Bayes classification. Naïve Bayes classification methods are based on Bayes Theorem, developed by the Reverend Thomas Bayes.¹ Bayes Theorem updates our knowledge about the data parameters by combining our previous knowledge (called the prior distribution) with new information obtained from observed data, resulting in updated parameter knowledge (called the posterior distribution).

8.2 BAYES THEOREM

Consider a data set made up of two predictors X = X₁, X₂ and a response variable Y, where the response variable takes one of three possible class values: y₁, y₂, and y₃ Our objective is to identify which of y₁, y₂, and y₃ is the most likely for a particular combination of predictor variable values. Let us call this most likely combination X^* = {X₁ = x₁, X₂ = x₂}.

We can use Bayes Theorem to identify which class is the most likely for a particular combination of predictor variable values by:

1. calculating the posterior probability for each of y₁, y₂, and y₃, for the combination of predictors x₁ and x₂ and
2. selecting the value of y with the highest posterior probability.

Let y^* be one of the three potential values of Y. Bayes Theorem tells us:

(8.1)

Now, p(Y = y^*) represents the ...