book

Machine Learning Foundations, Volume 1: Supervised Learning

Name: Machine Learning Foundations, Volume 1: Supervised Learning
Author: Roi Yehoshua
ISBN: 9780135337851

by Roi Yehoshua

January 2026

Intermediate to advanced

880 pages

25h 46m

English

Addison-Wesley Professional

Read now

Unlock full access

Cover Page
About This eBook
Title Page
Copyright Page
Contents
Preface
Organization of the BookWho This Book Is ForAbout the First VolumeAbout the ExercisesAbout the CodeAbout the DatasetsNotation and ConventionsAcknowledgmentsSupport and Feedback
About the Author
Chapter 1. Introduction to Machine Learning
1.1 Formal Definition1.2 Types of Machine Learning1.3 Related Fields1.4 Brief History1.5 Machine Learning Applications1.6 Limitations of Machine Learning1.7 Ethical Considerations1.8 Software Libraries1.9 Common Datasets1.10 Current Trends and Future Directions1.11 Summary
Chapter 2. Supervised Machine Learning
2.1 Formal Definition2.2 Machine Learning Models2.3 The Data-Generating Process2.4 Generalization Error and Empirical Risk Minimization2.5 Parameter Estimation2.6 The Bias–Variance Tradeoff2.7 Building a Machine Learning Model2.8 Challenges in Supervised Learning2.9 Summary2.10 Exercises
Chapter 3. Introduction to Scikit-Learn
3.1 Main Features3.2 Installation3.3 The Estimator API3.4 Typical Workflow of Building a Model3.5 Example: Iris Classification3.6 Pipelines3.7 Cross-Validation3.8 Hyperparameter Tuning3.9 Data Preparation3.10 Data Preprocessing3.11 Building Your Own Estimators3.12 Summary3.13 Exercises

Chapter 4. Linear Regression
4.1 Formal Definitions and Notations4.2 Ordinary Least Squares (OLS)4.3 Simple Linear Regression4.4 Multiple Linear Regression4.5 Regression Evaluation Metrics4.6 Example: Predicting Housing Prices4.7 Linear Regression Assumptions4.8 Gradient Descent for Linear Regression4.9 Alternative Loss Functions4.10 Nonlinear Regression4.11 Regularized Linear Regression4.12 (*) Bayesian Linear Regression4.13 Summary4.14 Exercises
Chapter 5. Logistic Regression
5.1 Classification Problems5.2 The Logistic Regression Model5.3 Training a Logistic Regression Model5.4 Logistic Regression in Scikit-Learn5.5 Classification Evaluation Metrics5.6 Learning from Imbalanced Data5.7 Multi-Class Extensions for Binary Classifiers5.8 Multinomial Logistic Regression5.9 (*) Generalized Linear Models5.10 Summary5.11 Exercises
Chapter 6. K-Nearest Neighbors
6.1 K-Nearest Neighbors Classification6.2 Choosing the Number of Neighbors6.3 Distance Metrics and Similarity Measures6.4 (*) KNN Classification Error Bounds6.5 (*) Efficient Data Structures for KNN6.6 The Curse of Dimensionality6.7 Radius-based Nearest Neighbors6.8 K-Nearest Neighbors Regression6.9 Variants of KNN6.10 Applications of KNN6.11 Summary6.12 Exercises
Chapter 7. Naive Bayes
7.1 The Naive Bayes Model7.2 Event Models in Naive Bayes7.3 Naive Bayes Classifiers in Scikit-Learn7.4 (*) Linear and Quadratic Discriminant Analysis7.5 Introduction to Natural Language Processing7.6 Document Classification Example7.7 Bayesian Networks7.8 (*) Probabilistic Graphical Models7.9 Summary7.10 Exercises
Chapter 8. Decision Trees
8.1 Decision Tree Definition8.2 Decision Tree Construction8.3 Tree Pruning8.4 Decision Tree Algorithms8.5 Decision Trees in Scikit-Learn8.6 Regression Trees8.7 Multi-Output Problems8.8 Oblique Decision Trees8.9 Summary8.10 Exercises
Chapter 9. Ensemble Methods
9.1 Introduction and Motivation9.2 Types of Ensemble Methods9.3 Using Different Learning Algorithms9.4 Bagging Ensembles9.5 Random Forests9.6 Boosting Ensembles9.7 AdaBoost9.8 Gradient Boosting9.9 Stacking Ensembles9.10 Summary9.11 Exercises
Chapter 10. Gradient Boosting Libraries
10.1 XGBoost10.2 LightGBM10.3 CatBoost10.4 The Higgs Boson Machine Learning Challenge10.5 Summary10.6 Exercises
Chapter 11. Support Vector Machines
11.1 Hard-Margin SVM11.2 Soft-Margin SVM11.3 Kernel Methods11.4 Nonlinear SVM11.5 ν-SVM11.6 Support Vector Regression (SVR)11.7 (*) Efficient Methods for Training SVMs11.8 Model Calibration11.9 Summary11.10 Exercises
Chapter 12. Summary and Additional Resources
12.1 Supervised Learning Summary12.2 Choosing a Learning Algorithm12.3 Conducting Machine Learning Experiments12.4 Research in Machine Learning12.5 Machine Learning Competitions12.6 Additional Resources12.7 Looking Ahead to Volume II12.8 Exercises
Bibliography
Index

Content preview from Machine Learning Foundations, Volume 1: Supervised Learning

Chapter 10 Gradient Boosting Libraries

This chapter explores three popular gradient boosting algorithms: XGBoost, LightGBM, and CatBoost, which introduce innovative techniques to enhance the performance, scalability, and robustness of gradient boosting. Each algorithm is supported by a dedicated Python library that provides highly optimized implementations and advanced features such as parallel or GPU-based training. These libraries have set new standards for efficiency and accuracy in supervised learning and are capable of handling massive datasets with millions of samples.

XGBoost is widely recognized for its predictive performance and scalability, making it a popular choice in data science competitions and real-world applications. LightGBM ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Hands-On Machine Learning with Scikit-Learn and PyTorch

Publisher Resources

ISBN: 9780135337851

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Machine Learning Foundations, Volume 1: Supervised Learning

by Roi Yehoshua

Chapter 10

Gradient Boosting Libraries

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.