Skip to Content
Reinforcement Learning with TensorFlow
book

Reinforcement Learning with TensorFlow

by Sayon Dutta
April 2018
Intermediate to advanced content levelIntermediate to advanced
334 pages
10h 18m
English
Packt Publishing
Content preview from Reinforcement Learning with TensorFlow

Why do we use xavier initialization?

The following factors call for the application of xavier initialization:

  • If the weights in a network start very small, most of the signals will shrink and become dormant at the activation function in the later layers

  • If the weights start very large, most of the signals will massively grow and pass through the activation functions in the later layers

Thus, xavier initialization helps in generating optimal weights, such that the signals are within optimal range, thereby minimizing the chances of the signals getting neither too small nor too large.

The derivation of the preceding formula is beyond the scope of this book. Feel free to search here (http://andyljones.tumblr.com/post/110998971763/an-explanation-of-xavier-initialization ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Deep Learning with TensorFlow - Second Edition

Deep Learning with TensorFlow - Second Edition

Giancarlo Zaccone, Vihan Jain, Md. Rezaul Karim, Motaz Saad
Deep Learning with TensorFlow 2 and Keras - Second Edition

Deep Learning with TensorFlow 2 and Keras - Second Edition

Antonio Gulli, Dr. Amita Kapoor, Sujit Pal

Publisher Resources

ISBN: 9781788835725Supplemental Content