Machine Learning for OpenCV 4 - Second Edition

Handling the missing data

Another common requirement in feature engineering is the handling of missing data. For example, we might have a dataset that looks like this:

In [11]: from numpy import nan...      X = np.array([[ nan, 0,   3 ],...                    [ 2,   9,  -8 ],...                    [ 1,   nan, 1 ],...                    [ 5,   2,   4 ],...                    [ 7,   6,  -3 ]])

Most machine learning algorithms cannot handle the Not a Number (NAN) values (nan in Python). Instead, we first have to replace all of the nan values with some appropriate fill values. This is known as the imputation of missing values.

Three different strategies to impute missing values are offered by scikit-learn:

• mean: Replaces all of the nan values with a mean value along a specified axis of the matrix (default: axis = 0)
• median: Replaces all ...