The problem with univariate selection is the likelihood of selecting a subset containing redundant information, whereas our interest is to get a minimum set that works with our predictor algorithm. In this case, recursive elimination could help provide the answer.
By running the following script, you'll find the reproduction of a problem that is quite challenging and which you may also often come across in datasets of different cases and variable sizes:
In: from sklearn.model_selection import train_test_split X, y = make_classification(n_samples=100, n_features=100, n_informative=5, n_redundant=2, random_state=101) X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=101) In: from ...
