Machine Learning Guide for Oil and Gas Using Python

Extreme gradient boosting

Extreme gradient boosting, also known as XGBoost, developed by Tianqi Chen (Chen & Guestrin, 2016), is another type of ensemble supervised ML algorithm used for both classification and regression problems. XGBoost is a type of gradient boosting method and is different from a gradient boosting model as follows:

• - XGBoost uses L₁ and L₂ regularization which helps with model generalization and overfitting reduction.
• - As previously discussed under the gradient boosting section, the error rate of a gradient boosting model is used to calculate the gradient, which is essentially the partial derivative of the loss function. In contrast, XGBoost uses the second partial derivative of the loss function. Using the second partial derivative ...