Machine Learning Algorithms - Second Edition

Multinomial Naive Bayes

A multinomial distribution is useful to model feature vectors where each value represents, for example, the number of occurrences of a term or its relative frequency. If the feature vectors have n elements and each of them can assume k different values with probability p_k, then:

The conditional probabilities P(x^_(i)|y_j) are computed with a frequency count (which corresponds to applying a maximum likelihood approach), but in this case, it's important to consider also a correction parameter α (called Laplace or Lidstone smoothing factor), to avoid null probabilities:

In the previous expression, the numerator is analogous ...