Python Feature Engineering Cookbook

How it works...

In this recipe, we replaced the categories with the WoE using pandas and Feature-engine.

We first loaded the dataset and divided it into train and test sets. Next, we calculated the mean target value per category using pandas' groupby() method over the categorical A1 variable and pandas' mean() method over the target A16 variable. This is equivalent to the probability of the target being 1, that is, p(1), per category. The output of these operations was a pandas Series with the categories as indices and the target mean as values. Next, we subtracted this pandas Series from 1 to create another pandas Series with the probability of the target being 0, that is, p(0), per category. Next, we created a third pandas Series with the ...