Skip to Main Content
Programming Machine Learning
book

Programming Machine Learning

by Paolo Perrotta
March 2020
Beginner to intermediate content levelBeginner to intermediate
342 pages
8h 38m
English
Pragmatic Bookshelf
Content preview from Programming Machine Learning

Initializing the Weights

Let me wax nostalgic about perceptrons for a moment. Back in Part I of this book, weight initialization was a quick job: we just set all the weights to 0. By contrast, weight initialization in a neural network comes with a hard-to-spot pitfall. Let’s describe that pitfall, and see how to walk around it.

Fearful Symmetry

Here is one rule to keep in mind: never initialize all the weights in a neural network with the same value. The reason for that recommendation is subtle, and comes from the matrix multiplications in the network. For example, look at this matrix multiplication:

images/training/symmetry.png

You don’t need to remember the details of matrix ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Mastering Machine Learning Algorithms - Second Edition

Mastering Machine Learning Algorithms - Second Edition

Giuseppe Bonaccorso
Practical Machine Learning for Computer Vision

Practical Machine Learning for Computer Vision

Valliappa Lakshmanan, Martin Görner, Ryan Gillard

Publisher Resources

ISBN: 9781680507706Errata Page