How to do it...

  1. Load the gradient boosting algorithm and random grid search:
from sklearn.ensemble import GradientBoostingRegressorfrom sklearn.model_selection import RandomizedSearchCV
  1. Create a parameter distribution for the gradient boosting trees:
param_dist = {'max_features' : ['log2',1.0], 'max_depth' : [3, 5, 7, 10], 'min_samples_leaf' : [2, 3, 5, 10], 'n_estimators': [50, 100], 'learning_rate' : [0.0001,0.001,0.01,0.05,0.1,0.3], 'loss' : ['ls','huber'] }
  1. Run the grid search to find the best parameters. Perform a randomized search with 30 iterations:
pre_gs_inst = RandomizedSearchCV(GradientBoostingRegressor(warm_start=True), param_distributions = param_dist, cv=3, n_iter = 30, n_jobs=-1)pre_gs_inst.fit(X_train, y_train)
  1. Now ...

Get scikit-learn Cookbook - Second Edition now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.