book

Machine Learning Algorithms in Depth

by Vadim Smolyakov

January 2025

Intermediate to advanced

328 pages

8h 28m

English

Manning Publications

Read now

Unlock full access

Who should read this bookHow this book is organizedAbout the codeliveBook discussion forum

1.1 Types of ML algorithms1.2 Why learn algorithms from scratch?1.3 Mathematical background1.4 Bayesian inference and deep learning1.4.1 Two main camps of Bayesian inference: MCMC and VI1.4.2 Modern deep learning algorithms1.5 Implementing algorithms1.5.1 Data structures1.5.2 Problem-solving paradigmsSummary
2.1 Introduction to Markov chain Monte Carlo2.1.1 Posterior distribution of coin flips2.1.2 Markov chain for page rank2.2 Estimating pi2.3 Binomial tree model2.4 Self-avoiding random walk2.5 Gibbs sampling2.6 Metropolis-Hastings sampling2.7 Importance sampling2.8 ExercisesSummary
3.1 KL variational inference3.2 Mean-field approximation3.3 Image denoising in an Ising model3.4 MI maximization3.5 ExercisesSummary
4.1 Data structures4.1.1 Linear4.1.2 Nonlinear4.1.3 Probabilistic4.2 Problem-solving paradigms4.2.1 Complete search4.2.2 Greedy4.2.3 Divide and conquer4.2.4 Dynamic programming4.3 ML research: Sampling methods and variational inference4.4 ExercisesSummary
5.1 Introduction to classification5.2 Perceptron5.3 Support vector machine5.4 Logistic regression5.5 Naive Bayes5.6 Decision tree (CART)5.7 ExercisesSummary
6.1 Introduction to regression6.2 Bayesian linear regression6.3 Hierarchical Bayesian regression6.4 KNN regression6.5 Gaussian process regression6.6 ExercisesSummary
7.1 Markov models7.1.1 Page rank algorithm7.1.2 Hidden Markov models7.2 Imbalanced learning7.2.1 Undersampling strategies7.2.2 Oversampling strategies7.3 Active learning7.3.1 Query strategies7.4 Model selection: Hyperparameter tuning7.4.1 Bayesian optimization7.5 Ensemble methods7.5.1 Bagging7.5.2 Boosting7.5.3 Stacking7.6 ML research: Supervised learning algorithms7.7 ExercisesSummary
8.1 Dirichlet process K-means8.2 Gaussian mixture models8.2.1 Expectation maximization (EM) algorithm8.3 Dimensionality reduction8.3.1 Principal component analysis8.3.2 t-SNE manifold learning on images8.4 ExercisesSummary
9.1 Latent Dirichlet allocation9.1.1 Variational Bayes9.2 Density estimators9.2.1 Kernel density estimator9.2.2 Tangent portfolio optimization9.3 Structure learning9.3.1 Chow-Liu algorithm9.3.2 Inverse covariance estimation9.4 Simulated annealing9.5 Genetic algorithm9.6 ML research: Unsupervised learning9.7 ExercisesSummary
10.1 Multilayer perceptron10.2 Convolutional neural nets10.2.1 LeNet on MNIST10.2.2 ResNet image search10.3 Recurrent neural nets10.3.1 LSTM sequence classification10.3.2 Multi-input model10.4 Neural network optimizers10.5 ExercisesSummary
11.1 Autoencoders11.1.1 VAE anomaly detection in time series11.2 Amortized variational inference11.2.1 Mixture density networks11.3 Attention and transformers11.4 Graph neural networks11.5 ML research: Deep learning11.6 ExercisesSummary
A.1 Competitive programmingA.2 Recommended booksA.3 Research conferencesA.3.1 Machine learningA.3.2 Computer visionA.3.3 Natural language processingA.3.4 Theoretical computer science

Content preview from Machine Learning Algorithms in Depth

10 Fundamental deep learning algorithms

This chapter covers

Multilayer perceptron
Convolutional neural nets: LeNet on MNIST and ResNet image search
Recurrent neural nets: LSTM sequence classification and multi-input neural net
Neural network optimizers

In the previous chapter, we discussed selected unsupervised ML algorithms to help discover patterns in our data. In this chapter, we introduce deep learning algorithms. Deep learning algorithms are part of supervised learning, which we encountered in chapters 5, 6, and 7. Deep learning algorithms revolutionized the industry and enabled many research and business applications previously thought to be out of reach by classic ML algorithms. We’ll begin this chapter with the fundamentals, such ...