TensorFlow 2.0 Quick Start Guide

Gradient calculations for gradient descent algorithms

One of TenorFlow's great strengths is its ability to automatically compute gradients for use in gradient descent algorithms, which, of course, are a vital part of most machine learning models. TensorFlow offers a number of methods for gradient calculations.

There are four ways to automatically compute gradients when eager execution is enabled (they also work in graph mode):

1. tf.GradientTape: Context records computations so that you can call tf.gradient() to get the gradients of any tensor computed while recording with respect to any trainable variable
2. tfe.gradients_function(): Takes a function (say f()) and returns a gradient function (say fg()) that can compute the gradients of the outputs ...