Data Mining Algorithms: Explained Using R

Chapter 4

Naïve Bayes classifier

4.1 Introduction

The naïve Bayes classifier is one of the simplest approaches to the classification task that is still capable of providing reasonable accuracy. Whereas in many cases it cannot compete with much more refined algorithms, such as decision trees, it sometimes does not stay far behind, and it may be even superior for certain specific application domains, with text classification being the most prominent example. Its simplicity—conceptual, implementational, and computational—makes it easy and inexpensive to try besides or before more sophisticated classifiers.

dmr.data

Example 4.1.1

Examples illustrating the naïve Bayes classifier will use the ultra small weather and weatherc datasets from Examples 1.3.1 and 1.3.2, respectively. These datasets, as well as DMR packages required to run some of example code snippets, are loaded by the following R code.

library(dmr.stats)
library(dmr.util)
data(weather, package="dmr.data")
data(weatherc, package="dmr.data")

4.2 Bayes rule

Bayesian inference, of which the naïve Bayes classifier is a particularly simple example, is based on the Bayes rule that relates conditional and marginal probabilities. More exactly, it shows how the conditional (posterior) probability of an event can be calculated based on its marginal (prior) probability and the inverse conditional probability. For two events and , the rule can be written as

(4.1)

where

• is the prior probability of ,
• is the ...