Python Feature Engineering Cookbook

How it works...

In this recipe, we grouped infrequent categories using pandas and Feature-engine. 

We first loaded the dataset and divided it into train and test sets. Next, we determined the percentage of observations per category of the A7 variable. With pandas' value_counts() method, we counted the observations per category of A7 and then divided these values by the total number of observations, determined using Python's built-in len method, to obtain the percentage of observations per category.

Next, we created a function that took as arguments a dataframe, a categorical variable, and a tolerance, which is the minimum frequency for a category not to be considered rare. In the function, we used pandas' value_counts() method and Python's  ...