book

Machine Learning Algorithms in Depth

by Vadim Smolyakov

January 2025

Intermediate to advanced

328 pages

8h 28m

English

Manning Publications

Read now

Unlock full access

Who should read this bookHow this book is organizedAbout the codeliveBook discussion forum

1.1 Types of ML algorithms1.2 Why learn algorithms from scratch?1.3 Mathematical background1.4 Bayesian inference and deep learning1.4.1 Two main camps of Bayesian inference: MCMC and VI1.4.2 Modern deep learning algorithms1.5 Implementing algorithms1.5.1 Data structures1.5.2 Problem-solving paradigmsSummary
2.1 Introduction to Markov chain Monte Carlo2.1.1 Posterior distribution of coin flips2.1.2 Markov chain for page rank2.2 Estimating pi2.3 Binomial tree model2.4 Self-avoiding random walk2.5 Gibbs sampling2.6 Metropolis-Hastings sampling2.7 Importance sampling2.8 ExercisesSummary
3.1 KL variational inference3.2 Mean-field approximation3.3 Image denoising in an Ising model3.4 MI maximization3.5 ExercisesSummary
4.1 Data structures4.1.1 Linear4.1.2 Nonlinear4.1.3 Probabilistic4.2 Problem-solving paradigms4.2.1 Complete search4.2.2 Greedy4.2.3 Divide and conquer4.2.4 Dynamic programming4.3 ML research: Sampling methods and variational inference4.4 ExercisesSummary
5.1 Introduction to classification5.2 Perceptron5.3 Support vector machine5.4 Logistic regression5.5 Naive Bayes5.6 Decision tree (CART)5.7 ExercisesSummary
6.1 Introduction to regression6.2 Bayesian linear regression6.3 Hierarchical Bayesian regression6.4 KNN regression6.5 Gaussian process regression6.6 ExercisesSummary
7.1 Markov models7.1.1 Page rank algorithm7.1.2 Hidden Markov models7.2 Imbalanced learning7.2.1 Undersampling strategies7.2.2 Oversampling strategies7.3 Active learning7.3.1 Query strategies7.4 Model selection: Hyperparameter tuning7.4.1 Bayesian optimization7.5 Ensemble methods7.5.1 Bagging7.5.2 Boosting7.5.3 Stacking7.6 ML research: Supervised learning algorithms7.7 ExercisesSummary
8.1 Dirichlet process K-means8.2 Gaussian mixture models8.2.1 Expectation maximization (EM) algorithm8.3 Dimensionality reduction8.3.1 Principal component analysis8.3.2 t-SNE manifold learning on images8.4 ExercisesSummary
9.1 Latent Dirichlet allocation9.1.1 Variational Bayes9.2 Density estimators9.2.1 Kernel density estimator9.2.2 Tangent portfolio optimization9.3 Structure learning9.3.1 Chow-Liu algorithm9.3.2 Inverse covariance estimation9.4 Simulated annealing9.5 Genetic algorithm9.6 ML research: Unsupervised learning9.7 ExercisesSummary
10.1 Multilayer perceptron10.2 Convolutional neural nets10.2.1 LeNet on MNIST10.2.2 ResNet image search10.3 Recurrent neural nets10.3.1 LSTM sequence classification10.3.2 Multi-input model10.4 Neural network optimizers10.5 ExercisesSummary
11.1 Autoencoders11.1.1 VAE anomaly detection in time series11.2 Amortized variational inference11.2.1 Mixture density networks11.3 Attention and transformers11.4 Graph neural networks11.5 ML research: Deep learning11.6 ExercisesSummary
A.1 Competitive programmingA.2 Recommended booksA.3 Research conferencesA.3.1 Machine learningA.3.2 Computer visionA.3.3 Natural language processingA.3.4 Theoretical computer science

Content preview from Machine Learning Algorithms in Depth

preface

Welcome to Machine Learning Algorithms in Depth! The idea of writing this book came to me during my graduate school years. At the time, I was switching majors from wireless communications to machine learning and found that one constant during this transition was my fascination with algorithms. I wanted to study this subject in depth and really understand how to derive, implement, and analyze algorithms from the first principles. I was fortunate to have found a research home in the Sensing, Learning, and Inference group at MIT CSAIL, where I was exposed to a wide variety of machine learning applications centered on Bayesian inference. At the same time, the field of deep learning was rapidly evolving, and I found myself training and ...