book

XGBoost for Regression Predictive Modeling and Time Series Analysis

Name: XGBoost for Regression Predictive Modeling and Time Series Analysis
ISBN: 9781805123057

by Partha Pritam Deka, Joyce Weiner

December 2024

Beginner to intermediate

308 pages

English

Packt Publishing

Read now

Unlock full access

XGBoost for Regression Predictive Modeling and Time Series Analysis
Foreword
Contributors
About the authors
About the reviewers
Preface
Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesConventions usedGet in touchShare Your ThoughtsDownload a free PDF copy of this book
Part 1:Introduction to Machine Learning and XGBoost with Case Studies
Chapter 1: An Overview of Machine Learning, Classification, and Regression
Fundamentals of machine learningSupervised and unsupervised learningClassification and regression decision tree modelsEnsembled models: bagging versus boostingData preparation and data engineeringData cleaningFeature engineeringSummary
Chapter 2: XGBoost Quick Start Guide with an Iris Data Case Study
Technical requirementsDownloading and installing the XGBoost packageIngesting and exploring dataIngesting the Iris datasetExploring the dataset by making graphsLooking for relationships with x-y plotsPreparing data for modelingSetting up and training XGBoostUsing XGBoost to make a predictionWhat’s next?Summary
Chapter 3: Demystifying the XGBoost Paper
Examining the paper—XGBoost: A Scalable Tree Boosting System—at a high levelExploring the features and benefits of XGBoostGradient-boosted treesProblems to tackle that would improve the gradient-boosted tree algorithmUnderstanding the XGBoost algorithmAddressing problems – sparse data, overfittingImprovements to address memory limitations and speed up computationComparing XGBoost with other ensemble decision tree modelsXGBoost compared with random forest modelsWhen to use gradient-boosted treesSummary
Chapter 4: Adding on to the Quick Start – Switching out the Dataset with a Housing Data Case Study
Technical requirementsSwitching the quick start code to a different datasetDownloading the housing datasetExploring the dataset by making graphsPreparing data for predictive modelingXGBoost predictive model settings and model trainingMaking a prediction using XGBoostComparing model parameters for the Iris and housing datasetsSummary

Part 2: Practical Applications – Data, Features, and Hyperparameters
Chapter 5: Classification and Regression Trees, Ensembles, and Deep Learning Models – What’s Best for Your Data?
Technical requirementsWhen to use ensemble models versus single CART modelsComparing models with the housing datasetComparing XGBoost to linear regressionWhen to use linear regressionComparing XGBoost to CARTWhen to use CARTComparing XGBoost to gradient boosting and random forest modelsWhen to use ensemble tree-based modelsWhen to use simple models versus ensemble tree modelsWhen to use deep learning models versus decision-tree-based modelsSetting XGBoost regression hyperparametersResults from comparing models with the housing datasetSummary
Chapter 6: Data Cleaning, Imbalanced Data, and Other Data Problems
Technical requirementsReal-life data is never cleanWhat to look for when exploring dataData-cleaning methodsCleaning continuous dataCleaning ordinal and nominal dataCorrecting spelling in a columnReformatting date dataHandling imbalanced dataCorrecting imbalanced data by sampling the datasetEnhancing data for a class by imputing valuesSummary
Chapter 7: Feature Engineering
Technical requirementsA review of exploratory data analysisDataset and data descriptionPerforming EDAPerforming feature engineering on the house prices datasetSeparating the dataset into training and testing setsHandling missing values for categorical variablesHandling missing values for numerical variablesCommon feature engineering techniques for numerical featuresPerforming a log transformLog transformation rationaleHandling skewed variables by transforming them into binary variablesScaling featuresCommon feature engineering techniques for temporal featuresHandling temporal variablesDeriving time differencesCommon feature engineering techniques for categorical featuresUnderstanding categorical variablesOrder mappings – capturing order or related qualityIdentifying rare labels for the rest of the categorical/nominal variablesEncoding categorical variables using a simple techniqueSummary
Chapter 8: Encoding Techniques for Categorical Features
Technical requirementsWhy is categorical encoding necessary?One-hot encodingOne-hot encoding with frequent categoriesInteger encodingCount or frequency encodingOrdered integer encodingMean encodingProbability ratio encodingWeight of Evidence (WoE) encodingEngineering rare categoriesSummary
Chapter 9: Using XGBoost for Time Series Forecasting
Technical requirementsTime series data – characteristics and componentsWhy use XGBoost for time series forecasting?Preparing time series data for XGBoostCreating lag featuresAdding date-based featuresRolling statisticsPractical considerations for handling time series dataTraining the XGBoost modelEvaluating the modelForecasting future valuesWhat is iterative forecasting?Forecasting future time stepsPreparing input data for forecastingVisualizing the forecastImproving future predictionsSummaryReferences
Chapter 10: Model Interpretability, Explainability, and Feature Importance with XGBoost
Technical requirementsWhy interpretability and explainability matterImplementing XGBoost’s feature importanceExploring SHAP for model interpretationWhy SHAP is usefulUsing SHAP with the California housing datasetImplementing LIME for model interpretationWhy use LIME?Using LIME to interpret XGBoost predictionsLIME versus SHAPApplying ELI5 for model interpretationWhy use ELI5?Explaining an XGBoost model with ELI5Exploring PDPs for model interpretationWhy use PDPs?Explaining gradient boosting model features with PDPsSummary
Part 3: Model Evaluation Metrics and Putting Your Model into Production
Chapter 11: Metrics for Model Evaluations and Comparisons
Technical requirementsWorking with the XGBoost APIUsing the native APIUsing the scikit-learn APIComparing the scikit-learn and native Python APIsEvaluating model performance of classification and regression modelsBinary classificationBinary classifier metrics – accuracy, precision, recall, and F1 scoreGraphical analysis of binary classification, ROC-AUC curves, and confusion matrixMulti-class classifier metrics – micro, weighted, and macro averagesMetrics for regression RMSE R2Evaluating how well a model fits the data summaryModel performance, training, and inference timeTuning XGBoost hyperparameters to improve model fit and efficiencyWhat is hyperparameter tuning?Key XGBoost hyperparametersPractical steps for tuningSummary
Chapter 12: Managing a Feature Engineering Pipeline in Training and Inference
Technical requirementsUsing pipelines to avoid data leaksIntroducing scikit-learn’s PipelineImplementing a data pipeline in PythonTime series feature engineering in a pipelineRegression feature engineering in a pipelineSummary
Chapter 13: Deploying Your XGBoost Model
Technical requirementsTraining a model using multithreaded and distributed computing with XGBoostUsing XGBoost’s multithreaded featuresTraining with distributed computePackaging a model for production deploymentTurning your model into an application with StreamlitDeploying the applicationCreating APIs to call your modelCreating an API for your model using FlaskUsing Docker to put your model into a containerDeploying a model using containersConsiderations when deploying a container in the cloudServicing your model using the REST APIMonitoring model performanceDetermining when to retrainRetraining your modelSummary
Index
Why subscribe?
Other Books You May EnjoyPackt is searching for authors like youShare Your ThoughtsDownload a free PDF copy of this book

Overview

Dive into the world of predictive modeling with "XGBoost for Regression Predictive Modeling and Time Series Analysis." This book provides you with a comprehensive understanding of the XGBoost algorithm, from its core concepts to practical applications, including regression, classification, and time series forecasting.

What this Book will help me do

Understand the mathematical foundation and intuitive concepts of XGBoost.
Learn how to implement XGBoost with Python for real-world modeling tasks.
Master techniques for feature engineering and model evaluation.
Explore advanced topics like model interpretability using SHAP and LIME.
Develop skills to deploy and maintain predictive XGBoost models in production environments.

Author(s)

Partha Pritam Deka and Joyce Weiner bring their extensive expertise in machine learning and data analysis to help readers excel in predictive modeling. With a combined experience of 40 years in industry, they present complex concepts in a clear and approachable manner. Their goal is to make advanced analytics tools accessible and applicable for diverse users.

Who is it for?

This book is designed for data scientists, machine learning practitioners, and analysts looking to advance their predictive modeling skills. If you're familiar with Python and basic machine learning concepts, you'll find this book a practical guide. Perfect for professionals aiming to harness XGBoost for regression, classification, and time series analysis projects. Accomplish your data modeling goals efficiently and confidently.

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781805123057

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills