Hands-On Deep Learning Algorithms with Python

Nadam – adding NAG to ADAM

Nadam is another small extension of the Adam method. As the name suggests, here, we incorporate NAG into Adam. First, let's recall what we learned about in Adam.

We calculated the first and second moments as follows:

Then, we calculated the bias-corrected estimates of the first and second moments, as follows:

Our final ...