book

Machine Learning for Tabular Data

Name: Machine Learning for Tabular Data
ISBN: 9781633438545

by Luca Massaron, Mark Ryan

February 2025

Intermediate to advanced

504 pages

16h 6m

English

Manning Publications

Read now

Unlock full access

Machine Learning for Tabular Data
Copyright
dedication
contents
front matter
forewordprefaceacknowledgmentsabout this bookWho should read this book?How this book is organized: A roadmapAbout the codeliveBook discussion forumabout the authorsabout the cover illustration
Part 1. Introducing machine learning for tabular data
1 Understanding tabular data
1.1 What is tabular data?1.2 The world runs on tabular data1.3 Machine learning vs. deep learning1.4 What makes tabular data different?1.5 Generative AI and tabular dataSummary
2 Exploring tabular datasets
2.1 Row and column characteristics2.1.1 The ideal criteria for tabular rows2.1.2 The ideal criteria for tabular columns2.1.3 Representing rows and columns2.2 Pathologies and remedies2.2.1 Constant or quasi-constant columns2.2.2 Duplicated and highly collinear features2.2.3 Irrelevant features2.2.4 Missing data2.2.5 Rare categories2.2.6 Errors in data2.2.7 Leakage features2.3 Finding external and internal data2.3.1 Using pandas to access data stores2.3.2 Internet data2.3.3 Synthetic data2.4 Exploratory data analysis2.4.1 Loading the Auto MPG example dataset2.4.2 Examining labels, values, distributions2.4.3 Exploring bivariate and multivariate relationshipsSummary
3 Machine learning vs. deep learning
3.1 Predicting Airbnb prices in New York City3.1.1 The Airbnb NYC dataset3.1.2 Introduction to the code3.1.3 A deep learning solution using Keras3.1.4 Training features3.1.5 Comparing gradient boosting and deep learning solutions3.1.6 Conclusions3.2 Transparency3.2.1 Explainability3.2.2 Feature importance3.2.3 Conclusions3.3 Efficacy3.3.1 Evaluating performance3.4 Digging DeeperSummary
Part 2. Machine learning and gradient boosting for tabular data

4 Classical algorithms for tabular data
4.1 Introducing Scikit-learn4.1.1 Common features of Scikit-learn packages4.1.2 Common Scikit-learn interface4.1.3 Introduction to Scikit-learn pipelines4.2 Exploring and processing features of the Airbnb NYC dataset4.2.1 Dataset exploration4.2.2 Pipelines preparation4.3 Classical machine learning4.3.1 Linear and logistic regression4.3.2 Regularized methods4.3.3 Logistic regression4.3.4 Generalized linear methods4.3.5 Handling large datasets with stochastic gradient descent4.3.6 Choosing your algorithmSummary
5 Decision trees and gradient boosting
5.1 Introduction to tree-based methods5.1.1 Bagging and sampling5.1.2 Predicting with random forests5.1.3 Resorting to extremely randomized trees5.2 Gradient boosting5.2.1 How gradient boosting works5.2.2 Extrapolating with gradient boosting5.2.3 Explaining gradient boosting effectiveness5.3 Boosting in Scikit-learn5.3.1 Applying early stopping to avoid overfitting5.4 Using XGBoost5.4.1 XGBoost’s key parameters5.4.2 How XGBoost works5.4.3 Accelerating with histogram splitting5.4.4 Applying early stopping to XGBoost5.5 Introduction to LightGBM5.5.1 How LightGBM grows trees5.5.2 Gaining speed with exclusive feature bundling and gradient-based one-side sampling5.5.3 Applying early stopping to LightGBM5.5.4 Making XGBoost imitate LightGBM5.5.5 How LightGBM inspired Scikit-learnSummary
6 Advanced feature processing methods
6.1 Processing features6.1.1 Multivariate missing data imputation6.1.2 Handling missing data with GBDTs6.1.3 Target encoding6.1.4 Transforming numerical data6.2 Selecting features6.2.1 Stability selection for linear models6.2.2 Shadow features and Boruta6.2.3 Forward and backward selection6.3 Optimizing hyperparameters6.3.1 Searching systematically6.3.2 Using random trials6.3.3 Reducing the computational burden6.3.4 Extending your search by Bayesian methods6.3.5 Manually setting hyperparameters6.4 Mastering gradient boosting6.4.1 Deciding between XGBoost and LightGBM6.4.2 Exploring tree structures6.4.3 Speeding up by GBDTs and compilingSummary
7 An end-to-end example using XGBoost
7.1 Preparing and exploring your data7.1.1 Using generative AI to help prepare data7.1.2 Getting and preparing your data7.1.3 Engineering more complex features7.1.4 Finalizing your data7.1.5 Exploring and fixing your data7.1.6 Exploring your target7.2 Building and optimizing your model7.2.1 Preparing a cross-validation strategy7.2.2 Preparing your pipeline7.2.3 Building a baseline model7.2.4 Building a first tentative model7.2.5 Optimizing your model7.2.6 Training the final model7.3 Explaining your model with SHAPSummary
Part 3. Deep learning for tabular data
8 Getting started with deep learning with tabular data
8.1 The deep learning with tabular data stack8.2 PyTorch with fastai8.2.1 Reviewing the key code aspects of the fastai solution8.2.2 Comparing the fastai solution with the Keras solution8.3 PyTorch with TabNet8.3.1 Key code aspects of the TabNet solution8.3.2 Comparing the TabNet solution with the Keras solution8.4 PyTorch with Lightning Flash8.4.1 The key code aspects of the Lightning Flash solution8.4.2 Comparing the Lightning Flash solution with the Keras solution8.5 Overall comparison of the stacks8.6 The stacks we didn’t exploreSummary
9 Deep learning best practices
9.1 Introduction to the Kuala Lumpur real estate dataset9.2 Processing the dataset9.2.1 Processing Bathrooms, Car Parks, Furnishing, Property Type, and Location columns9.2.2 Processing the Price column9.2.3 Processing the Rooms column9.2.4 Processing the Size column9.3 Defining the deep learning model9.3.1 Contrasting the custom layer and Keras preprocessing layer approaches9.3.2 Examining the code for model definition using Keras preprocessing layers9.4 Training the deep learning model9.4.1 Cross-validation in the training process9.4.2 Regularization in the training process9.4.3 Normalization in the training process9.5 Exercising the deep learning model9.5.1 Rationale for exercising the trained model on some new data points9.5.2 Exercising the trained model on some new data pointsSummary
10 Model deployment
10.1 A simple web deployment10.1.1 Overview of web deployment10.1.2 The Flask server module10.1.3 The home.html page10.1.4 The show-prediction.html page10.1.5 Exercising the web deployment10.2 Public clouds and machine learning operations10.3 Getting started with Google Cloud10.3.1 Accessing Google Cloud for the first time10.3.2 Creating a Google Cloud project10.3.3 Creating a Google Cloud Storage bucket10.4 Deploying a model in Vertex AI10.4.1 Uploading the model to a Cloud Storage bucket10.4.2 Importing the model to Vertex AI10.4.3 Deploying the model to an endpoint10.4.4 Initial test of the model deployment10.5 Using the Vertex AI deployment with Flask10.5.1 Setting up the Vertex AI SDK10.5.2 Updating the Flask server module to call the endpoint10.5.3 Benefits of deploying a model to an endpoint10.6 Gemini for Google Cloud: Generative AI assistance in Google Cloud10.6.1 Setting up Gemini for Google Cloud10.6.1 Using Gemini for Google Cloud to answer questions about Google CloudSummary
11 Building a machine learning pipeline
11.1 Introduction to ML pipelines11.1.1 Three kinds of pipelines11.1.2 Overview of Vertex AI ML pipelines11.2 ML pipeline preparation steps11.2.1 Creating a service account for the ML pipeline11.2.2 Creating a service account key11.2.3 Granting the service account access to the Compute Engine default service account11.2.4 Introduction to Cloud Shell11.2.5 Uploading the service account key11.2.6 Uploading the cleaned-up dataset to a Google Cloud Storage bucket11.2.7 Creating a Vertex AI managed dataset11.3 Defining the ML pipeline11.3.1 Local implementation vs. ML pipeline11.3.2 Introduction to containers11.3.3 Benefits of using containers in an ML pipeline11.3.4 Introduction to adapting code to run in a container11.3.5 Updating the training code to work in a container11.3.6 The pipeline script11.3.7 Testing the model trained in the pipeline11.4 Using generative AI to help create the ML pipeline11.4.1 Using Gemini for Google Cloud to answer questions about the ML pipeline11.4.2 Using Gemini for Google Cloud to generate code for the ML pipeline11.4.3 Using Gemini for Google Cloud to explain code for the ML pipeline11.4.4 Using Gemini for Google Cloud to summarize log entries11.4.5 Tuning a foundation model in Vertex AISummary
12 Blending gradient boosting and deep learning
12.1 Review of the gradient boosting solution from chapter 712.2 Selecting a deep learning solution12.3 Selected deep learning solution to the Tokyo Airbnb problem12.4 Comparing the XGBoost and fastai solutions to the Tokyo Airbnb problem12.5 Ensembling the two solutions to the Tokyo Airbnb problem12.6 Overall comparison of gradient boosting and deep learningSummary
Appendix A. Hyperparameters for classical machine learning models
Appendix B. K-nearest neighbors and support vector machines
B.1 k-NNB.2 SVMsB.3 Using GPUs for machine learning
index

Content preview from Machine Learning for Tabular Data

5 Decision trees and gradient boosting

This chapter covers

Decision trees and their ensembles
Gradient boosting decision trees
Scikit-learn’s gradient boosting decision trees options
XGBoost algorithm and its innovations
How LightGBM algorithm works

So far, we have explored machine learning algorithms based on linear models because they can handle tabular problems from datasets consisting of a few rows and columns and find a way to scale to problems of millions of rows and many columns. In addition, linear models are fast to train and get predictions from. Moreover, they are relatively easy to understand, explain, and tweak. Linear models are also helpful because they present many concepts we will keep building on in the book, such as L1 and ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Architecting Data and Machine Learning Platforms

Publisher Resources

ISBN: 9781633438545Publisher Support Publisher Website Errata Page

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Machine Learning for Tabular Data

by Luca Massaron, Mark Ryan

5 Decision trees and gradient boosting

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.