Best practice 3 – maintaining the consistency of field values

In a dataset that already exists, or in one we collect from scratch, oftentimes we see different values representing the same meaning. For example, there are American, US, and U.S.A in the country field, and male and M in the gender field. It is necessary to unify or standardize values in a field. For example, we can only keep M and F in the gender field and replace other alternatives. Otherwise it will mess up the algorithms in later stages as different feature values will be treated differently even if they have the same meaning. It is also a great practice to keep track of what values are mapped to the default value of a field.

In addition, the format of values in the same field ...

Get Python Machine Learning By Example - Second Edition now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.