Python Feature Engineering Cookbook

How it works...

In this recipe, we determined the percentage of missing data for each variable in the Credit Approval Data Set and removed all observations with missing information to create a complete case dataset.

First, we loaded the data from a CSV file into a dataframe with the pandas read_csv() method. Next, we used the pandas isnull() and mean() methods to determine the percentage of missing observations for each variable. We discussed these methods in the Quantifying missing data recipe in Chapter 1, Foreseeing Variable Problems When Building ML Models. With pandas sort_values(), we ordered the variables from the one with the fewest missing values to the one with the most.

To remove observations with missing values in any of the variables, ...