book

Machine Learning Algorithms in Depth

by Vadim Smolyakov

January 2025

Intermediate to advanced

328 pages

8h 28m

English

Manning Publications

Read now

Unlock full access

Who should read this bookHow this book is organizedAbout the codeliveBook discussion forum

1.1 Types of ML algorithms1.2 Why learn algorithms from scratch?1.3 Mathematical background1.4 Bayesian inference and deep learning1.4.1 Two main camps of Bayesian inference: MCMC and VI1.4.2 Modern deep learning algorithms1.5 Implementing algorithms1.5.1 Data structures1.5.2 Problem-solving paradigmsSummary
2.1 Introduction to Markov chain Monte Carlo2.1.1 Posterior distribution of coin flips2.1.2 Markov chain for page rank2.2 Estimating pi2.3 Binomial tree model2.4 Self-avoiding random walk2.5 Gibbs sampling2.6 Metropolis-Hastings sampling2.7 Importance sampling2.8 ExercisesSummary
3.1 KL variational inference3.2 Mean-field approximation3.3 Image denoising in an Ising model3.4 MI maximization3.5 ExercisesSummary
4.1 Data structures4.1.1 Linear4.1.2 Nonlinear4.1.3 Probabilistic4.2 Problem-solving paradigms4.2.1 Complete search4.2.2 Greedy4.2.3 Divide and conquer4.2.4 Dynamic programming4.3 ML research: Sampling methods and variational inference4.4 ExercisesSummary
5.1 Introduction to classification5.2 Perceptron5.3 Support vector machine5.4 Logistic regression5.5 Naive Bayes5.6 Decision tree (CART)5.7 ExercisesSummary
6.1 Introduction to regression6.2 Bayesian linear regression6.3 Hierarchical Bayesian regression6.4 KNN regression6.5 Gaussian process regression6.6 ExercisesSummary
7.1 Markov models7.1.1 Page rank algorithm7.1.2 Hidden Markov models7.2 Imbalanced learning7.2.1 Undersampling strategies7.2.2 Oversampling strategies7.3 Active learning7.3.1 Query strategies7.4 Model selection: Hyperparameter tuning7.4.1 Bayesian optimization7.5 Ensemble methods7.5.1 Bagging7.5.2 Boosting7.5.3 Stacking7.6 ML research: Supervised learning algorithms7.7 ExercisesSummary
8.1 Dirichlet process K-means8.2 Gaussian mixture models8.2.1 Expectation maximization (EM) algorithm8.3 Dimensionality reduction8.3.1 Principal component analysis8.3.2 t-SNE manifold learning on images8.4 ExercisesSummary
9.1 Latent Dirichlet allocation9.1.1 Variational Bayes9.2 Density estimators9.2.1 Kernel density estimator9.2.2 Tangent portfolio optimization9.3 Structure learning9.3.1 Chow-Liu algorithm9.3.2 Inverse covariance estimation9.4 Simulated annealing9.5 Genetic algorithm9.6 ML research: Unsupervised learning9.7 ExercisesSummary
10.1 Multilayer perceptron10.2 Convolutional neural nets10.2.1 LeNet on MNIST10.2.2 ResNet image search10.3 Recurrent neural nets10.3.1 LSTM sequence classification10.3.2 Multi-input model10.4 Neural network optimizers10.5 ExercisesSummary
11.1 Autoencoders11.1.1 VAE anomaly detection in time series11.2 Amortized variational inference11.2.1 Mixture density networks11.3 Attention and transformers11.4 Graph neural networks11.5 ML research: Deep learning11.6 ExercisesSummary
A.1 Competitive programmingA.2 Recommended booksA.3 Research conferencesA.3.1 Machine learningA.3.2 Computer visionA.3.3 Natural language processingA.3.4 Theoretical computer science

Content preview from Machine Learning Algorithms in Depth

4 Software implementation

This chapter covers

Data structures: linear, nonlinear, and probabilistic
Problem-solving paradigms: complete search, greedy, divide and conquer, and dynamic programming
ML research: sampling methods and variational inference

In the previous chapters, we looked at two main camps of Bayesian inference: Markov chain Monte Carlo and variational inference. In this chapter, we review computer science concepts required for implementing algorithms from scratch. To write high-quality code, it’s important to have a good grasp of data structures and algorithm fundamentals. This chapter is designed to introduce common computational structures and problem-solving paradigms. Many of the concepts reviewed in this section are ...