Skip to Content
Reinforcement Learning with TensorFlow
book

Reinforcement Learning with TensorFlow

by Sayon Dutta
April 2018
Intermediate to advanced content levelIntermediate to advanced
334 pages
10h 18m
English
Packt Publishing
Content preview from Reinforcement Learning with TensorFlow

The vanishing gradient problem

The vanishing gradient problem is one of the problems associated with the training of artificial neural networks when the neurons present in the early layers are not able to learn because the gradients that train the weights shrink down to zero. This happens due to the greater depth of neural network, along with activation functions with derivatives resulting in low value.

Try the following steps:

  1. Create one hidden layer neural network
  2. Add more hidden layers, one by one

We observe the gradient with regards to all the nodes, and find that the gradient values get relatively smaller when we move from the later layers to the early layers. This condition worsens with the further addition of layers. This shows that ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Deep Learning with TensorFlow - Second Edition

Deep Learning with TensorFlow - Second Edition

Giancarlo Zaccone, Vihan Jain, Md. Rezaul Karim, Motaz Saad
Deep Learning with TensorFlow 2 and Keras - Second Edition

Deep Learning with TensorFlow 2 and Keras - Second Edition

Antonio Gulli, Dr. Amita Kapoor, Sujit Pal

Publisher Resources

ISBN: 9781788835725Supplemental Content