Building Machine Learning Systems with Python - Third Edition

Accounting for unseen words and other oddities

When we calculated the probabilities earlier, we actually cheated ourselves. We were not calculating the real probabilities, but only rough approximations by means of the fractions. We assumed that the training corpus would tell us the whole truth about the real probabilities. It did not. A corpus of only six tweets obviously cannot give us all the information about every tweet that has ever been written. For example, there certainly are tweets containing the word text in them. It is only that we have never seen them. Apparently, our approximation is very rough, and we should account for that. This is often done in practice with the so-called add-one smoothing.

Add-one smoothing is sometimes ...