O'Reilly logo

Machine Learning with Spark - Second Edition by Nick Pentreath, Manpreet Singh Ghotra, Rajdeep Dua

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Gradient boosted tree regression

Gradient boosted trees are ensembles of decision trees. Gradient boosted trees iteratively train decision trees to minimize the loss function. Gradient boosted trees handle categorical features, support multiclass, and don't require feature scaling.

Spark ML implements gradient boosted trees using the existing decision tree implementation. It supports both classification and regression.

Let's train the bike sharing dataset by splitting it into 80% training and 20% testing, use GBTRegressor with regression evaluator from Spark to build the model, and get evaluation metrics around the test data.

@transient lazy val logger = Logger.getLogger(getClass.getName) def gbtRegressionWithVectorFormat(vectorAssembler: ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required