Python Feature Engineering Cookbook

How it works...

In this recipe, we quantified and displayed the amount and percentage of missing data of a publicly available dataset.

To load data from the txt file into a dataframe, we used the pandas read_csv() method. To load only certain columns from the original data, we created a list with the column names and passed this list to the usecols argument of read_csv(). Then, we used the head() method to display the top five rows of the dataframe, along with the variable names and some of their values.

To identify missing observations, we used pandas isnull(). This created a boolean vector per variable, with each vector indicating whether the value was missing (True) or not (False) for each row of the dataset. Then, we used the pandas ...