Machine Learning, 2nd Edition

Organization — How to Use This Book

The ideal method of science is the study of the direct influence of one condition on another in experiments in which all other possible causes of variation are eliminated. Unfortunately, causes of variation often seem to be beyond control.

— Sewall Wright, Correlation and Causation, 1921

The core material of this book is laid out in Chapters 1 through 7. These chapters are intended to be read in order, as each chapter builds on the previous ones. Chapters 1 and 2 introduce problems that machine learning methods can solve, and also introduce the fundamental ideas and terminology used to describe these problems and their solutions. Chapter 3 gives a brief introduction to regression.² Chapter 4 presents many methods for classification, grouped according to how they approach the problem. Chapter 5 discusses bias-variance trade-off, a topic essential to understanding the design principles behind ensemble methods. Chapter 6 presents various ensemble methods, focusing on how each method can be understood as trading off bias for variance, or vice versa. Chapter 7 concludes the core material with methods for risk estimation and model selection. By the end of Chapter 7, the reader will have encountered many useful methods for approaching classification and regression problems, and (I hope) will appreciate how each method works and why each method approaches classification or regression the way it does.

After Chapter 7, the reader can select from ...