We can use L1 and L2 regularization to find optimal coefficients for our feature selection, just as we did with our tree-based models. Let's use a logistic regression model as our model-based selector and gridsearch across both the L1 and L2 norm:
# a new selector that uses the coefficients from a regularized logistic regression as feature importanceslogistic_selector = SelectFromModel(LogisticRegression())# make a new pipeline that uses coefficients from LogistisRegression as a feature rankerregularization_pipe = Pipeline([('select', logistic_selector), ('classifier', tree)])regularization_pipe_params = deepcopy(tree_pipe_params)# try l1 regularization and l2 regularization ...
