Building Machine Learning Systems with Python - Third Edition

Accounting for arithmetic underflows

There is yet another roadblock. In reality, we work with probabilities much smaller than the ones we have dealt with in the toy example. Typically, we also have many more than only two features, which we multiply with each other. This will quickly lead to the point where the floating-point accuracy provided by NumPy does not suffice any more:

>>> import numpy as np>>> # tell numpy to print out more digits (default is 8)>>> np.set_printoptions(precision=20) >>> np.array([2.48E-324]) array([ 4.94065645841246544177e-324])>>> np.array([2.47E-324]) array([ 0.])  

So, how probable is it that we will ever hit a number such as 2.47E-324? To answer this, we just need to imagine a likelihood for the conditional probabilities ...