book

Machine Learning Foundations, Volume 1: Supervised Learning

by Roi Yehoshua

September 2025

Intermediate to advanced

812 pages

23h 14m

English

Addison-Wesley Professional

Read now

Unlock full access

1.1 Formal Definition1.2 Types of Machine Learning1.3 Related Fields1.4 Brief History1.5 Machine Learning Applications1.6 Limitations of Machine Learning1.7 Ethical Considerations1.8 Software Libraries1.9 Common Datasets1.10 Current Trends and Future Directions1.11 Summary
2.1 Formal Definition2.2 Machine Learning Models2.3 The Data-Generating Process2.4 Maximum Likelihood Estimation2.5 Bayesian Statistics2.6 The Bias-Variance Tradeoff2.7 Building a Machine Learning Model2.8 Challenges in Supervised Learning2.9 Summary2.10 Exercises
3.1 Main Features3.2 Installation3.3 The Estimator API3.4 Typical Workflow of Building a Model3.5 Example: Iris Classification3.6 Pipelines3.7 Cross-Validation3.8 Hyperparameter Tuning3.9 Data Preparation3.10 Data Preprocessing3.11 Building Your Own Estimators3.12 Summary3.13 Exercises
4.1 Formal Definitions and Notations4.2 Ordinary Least Squares (OLS)4.3 Simple Linear Regression4.4 Multiple Linear Regression4.5 Regression Evaluation Metrics4.6 Example: Predicting House Prices4.7 Linear Regression Assumptions4.8 Gradient Descent4.9 Alternative Loss Functions4.10 Non-Linear Regression4.11 Regularized Linear Regression4.12 (*) Bayesian Linear Regression4.13 Summary4.14 Exercises

5.1 Classification Problems5.2 The Logistic Regression Model5.3 Training a Logistic Regression Model5.4 Logistic Regression in Scikit-Learn5.5 Classification Evaluation Metrics5.6 Learning from Imbalanced Data5.7 Multi-Class Extensions for Binary Classifiers5.8 Multinomial Logistic Regression5.9 (*) Generalized Linear Models5.10 Summary5.11 Exercises
6.1 K-Nearest Neighbors Classification6.2 Choosing the Number of Neighbors6.3 Distance Metrics and Similarity Measures6.4 (*) KNN Classification Error Bounds6.5 Efficient Data Structures for KNN6.6 The Curse of Dimensionality6.7 Radius-based Nearest Neighbors6.8 K-Nearest Neighbors Regression6.9 Variants of KNN6.10 Applications of KNN6.11. Summary6.12 Exercises
7.1 The Naive Bayes Model7.2 Event Models in Naive Bayes7.3 Naive Bayes Classifiers in Scikit-Learn7.4 (*) Linear and Quadratic Discriminant Analysis7.5 Introduction to Natural Language Processing7.6 Document Classification Example7.7 Bayesian Networks7.8 Summary7.9 Exercises
8.1 Decision Tree Definition8.2 Decision Tree Construction8.3 Tree Pruning8.4 Decision Tree Algorithms8.5 Decision Trees in Scikit-Learn8.6 Regression Trees8.7 Multi-Output Problems8.8 Oblique Decision Trees8.9 Summary8.10 Exercises
9.1 Introduction and Motivation9.2 Types of Ensemble Methods9.3 Using Different Learning Algorithms9.4 Bagging Ensembles9.5 Random Forests9.6 Boosting Ensembles9.7 AdaBoost9.8 Gradient Boosting9.9 Stacking Ensembles9.10 Summary9.11 Exercises
10.1 XGBoost10.2 LightGBM10.3 CatBoost10.4 The Higgs Boson Machine Learning Challenge10.5 Summary10.6 Exercises
11.1 Hard-Margin SVM11.2 Soft-Margin SVM11.3 Kernel Methods11.4 Non-Linear SVM11.5 ν-SVM11.6 Support Vector Regression (SVR)11.7 (*) Efficient Methods for Training SVMs11.8 Model Calibration11.9 Summary11.10 Exercises
12.1 Supervised Learning Summary12.2 Choosing a Learning Algorithm12.3 Conducting Machine Learning Experiments12.4 Research in Machine Learning12.5 Machine Learning Competitions12.6 Additional Resources12.7 Exercises

Content preview from Machine Learning Foundations, Volume 1: Supervised Learning

Chapter 4. Linear Regression

Linear regression is one of the most fundamental predictive models in machine learning and statistics. Its origins trace back to the early 19th century, when Legendre and Gauss used linear regression to predict the movement of the planets.

The main objective in regression analysis is to predict the value of a dependent variable based on one or more independent variables. Its applications span a wide range of fields, from forecasting stock prices using various economic indicators, to estimating crop yields based on climate data, to medical applications such as detecting tumors in CT scans.

In linear regression specifically, we assume that there is a linear relationship between the input features (such as economic indicators) ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Start your free trial

Machine Learning with Python Cookbook, 2nd Edition

Publisher Resources

ISBN: 9780135337851

Machine Learning Foundations, Volume 1: Supervised Learning

by Roi Yehoshua

Chapter 4. Linear Regression

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

You might also like

Machine Learning with Python Cookbook, 2nd Edition

Mathematics of Machine Learning

Python Machine Learning - Third Edition

Machine Learning Algorithms in Depth

Publisher Resources

Chapter 4. Linear Regression

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,and much more.

You might also like

Machine Learning with Python Cookbook, 2nd Edition

Mathematics of Machine Learning

Python Machine Learning - Third Edition

Machine Learning Algorithms in Depth

Publisher Resources

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.