Feature Engineering Made Easy

Linear model coefficients as another feature importance metric

We can use L1 and L2 regularization to find optimal coefficients for our feature selection, just as we did with our tree-based models. Let's use a logistic regression model as our model-based selector and gridsearch across both the L1 and L2 norm:

# a new selector that uses the coefficients from a regularized logistic regression as feature importanceslogistic_selector = SelectFromModel(LogisticRegression())# make a new pipeline that uses coefficients from LogistisRegression as a feature rankerregularization_pipe = Pipeline([('select', logistic_selector),  ('classifier', tree)])regularization_pipe_params = deepcopy(tree_pipe_params)# try l1 regularization and l2 regularization ...