book

Machine Learning Algorithms in Depth

Name: Machine Learning Algorithms in Depth
Author: Vadim Smolyakov
ISBN: 9781633439214

by Vadim Smolyakov

January 2025

Intermediate to advanced

328 pages

8h 28m

English

Manning Publications

Audiobook available

Read now

Unlock full access

Machine Learning Algorithms in Depth
copyright
dedication
contents
preface
acknowledgments
about this book
Who should read this bookHow this book is organizedAbout the codeliveBook discussion forum
about the author
about the cover illustration
Part 1 Introducing ML algorithms

1 Machine learning algorithms
1.1 Types of ML algorithms1.2 Why learn algorithms from scratch?1.3 Mathematical background1.4 Bayesian inference and deep learning1.4.1 Two main camps of Bayesian inference: MCMC and VI1.4.2 Modern deep learning algorithms1.5 Implementing algorithms1.5.1 Data structures1.5.2 Problem-solving paradigmsSummary
2 Markov chain Monte Carlo
2.1 Introduction to Markov chain Monte Carlo2.1.1 Posterior distribution of coin flips2.1.2 Markov chain for page rank2.2 Estimating pi2.3 Binomial tree model2.4 Self-avoiding random walk2.5 Gibbs sampling2.6 Metropolis-Hastings sampling2.7 Importance sampling2.8 ExercisesSummary
3 Variational inference
3.1 KL variational inference3.2 Mean-field approximation3.3 Image denoising in an Ising model3.4 MI maximization3.5 ExercisesSummary
4 Software implementation
4.1 Data structures4.1.1 Linear4.1.2 Nonlinear4.1.3 Probabilistic4.2 Problem-solving paradigms4.2.1 Complete search4.2.2 Greedy4.2.3 Divide and conquer4.2.4 Dynamic programming4.3 ML research: Sampling methods and variational inference4.4 ExercisesSummary
Part 2 Supervised learning
5 Classification algorithms
5.1 Introduction to classification5.2 Perceptron5.3 Support vector machine5.4 Logistic regression5.5 Naive Bayes5.6 Decision tree (CART)5.7 ExercisesSummary
6 Regression algorithms
6.1 Introduction to regression6.2 Bayesian linear regression6.3 Hierarchical Bayesian regression6.4 KNN regression6.5 Gaussian process regression6.6 ExercisesSummary
7 Selected supervised learning algorithms
7.1 Markov models7.1.1 Page rank algorithm7.1.2 Hidden Markov models7.2 Imbalanced learning7.2.1 Undersampling strategies7.2.2 Oversampling strategies7.3 Active learning7.3.1 Query strategies7.4 Model selection: Hyperparameter tuning7.4.1 Bayesian optimization7.5 Ensemble methods7.5.1 Bagging7.5.2 Boosting7.5.3 Stacking7.6 ML research: Supervised learning algorithms7.7 ExercisesSummary
Part 3 Unsupervised learning
8 Fundamental unsupervised learning algorithms
8.1 Dirichlet process K-means8.2 Gaussian mixture models8.2.1 Expectation maximization (EM) algorithm8.3 Dimensionality reduction8.3.1 Principal component analysis8.3.2 t-SNE manifold learning on images8.4 ExercisesSummary
9 Selected unsupervised learning algorithms
9.1 Latent Dirichlet allocation9.1.1 Variational Bayes9.2 Density estimators9.2.1 Kernel density estimator9.2.2 Tangent portfolio optimization9.3 Structure learning9.3.1 Chow-Liu algorithm9.3.2 Inverse covariance estimation9.4 Simulated annealing9.5 Genetic algorithm9.6 ML research: Unsupervised learning9.7 ExercisesSummary
Part 4 Deep learning
10 Fundamental deep learning algorithms
10.1 Multilayer perceptron10.2 Convolutional neural nets10.2.1 LeNet on MNIST10.2.2 ResNet image search10.3 Recurrent neural nets10.3.1 LSTM sequence classification10.3.2 Multi-input model10.4 Neural network optimizers10.5 ExercisesSummary
11 Advanced deep learning algorithms
11.1 Autoencoders11.1.1 VAE anomaly detection in time series11.2 Amortized variational inference11.2.1 Mixture density networks11.3 Attention and transformers11.4 Graph neural networks11.5 ML research: Deep learning11.6 ExercisesSummary
appendix A Further reading and resources
A.1 Competitive programmingA.2 Recommended booksA.3 Research conferencesA.3.1 Machine learningA.3.2 Computer visionA.3.3 Natural language processingA.3.4 Theoretical computer science
appendix B Answers to exercises
index

Content preview from Machine Learning Algorithms in Depth

5 Classification algorithms

This chapter covers

Introducing classification
The perceptron algorithm
The SVM algorithm
SGD logistic regression
The Bernoulli naive Bayes algorithm
The decision tree (CART) algorithm

In the previous chapter, we looked at the computer science fundamentals required to implement ML algorithms from scratch. In this chapter, we focus on supervised learning algorithms. Classification is a fundamental class of algorithms and is widely used in machine learning. We will derive from scratch and implement several selected classification algorithms to build our experience with fundamentals and motivate the design of new ML algorithms. The algorithms in this chapter were selected because they illustrate important algorithmic ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781633439214Publisher Support Publisher Website

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Machine Learning Algorithms in Depth

by Vadim Smolyakov