Skip to Main Content
Programming Machine Learning
book

Programming Machine Learning

by Paolo Perrotta
March 2020
Beginner to intermediate content levelBeginner to intermediate
342 pages
8h 38m
English
Pragmatic Bookshelf
Content preview from Programming Machine Learning

Understanding Batches

Mini-batch GD feels counter-intuitive. Why do smaller batches result in faster training? The answer is that they don’t: if anything, mini-batch GD is generally slower than batch GD at processing the whole training set because it calculates the gradient for each batch, rather than once for all the examples.

Even if mini-batch GD is slower, it tends to converge faster during the first iterations of training. In other words, mini-batch GD is slower at processing the training set, but it moves quicker toward the target, giving us that fast feedback we need. Let’s see how.

Twist That Path

To see why mini-batches converge faster, I visualized gradient descent on a small two-dimensional training set. As usual, you’ll find the ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Mastering Machine Learning Algorithms - Second Edition

Mastering Machine Learning Algorithms - Second Edition

Giuseppe Bonaccorso
Practical Machine Learning for Computer Vision

Practical Machine Learning for Computer Vision

Valliappa Lakshmanan, Martin Görner, Ryan Gillard

Publisher Resources

ISBN: 9781680507706Errata Page