July 2017
Beginner to intermediate
442 pages
10h 8m
English
Content preview from Statistics for Machine Learning
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
Start your free trial
Adaptive moment estimation - Adam
Adam is another method that computes adaptive learning rates for each parameter. In addition to storing an exponentially decaying average of past squared gradients like Adadelta and RMSprop, Adam also keeps an exponentially decaying average of past gradients, similar to momentum.
When you are in doubt, just use Adam!