Hands-On Deep Learning Algorithms with Python

Gradients with respect to gates

Calculating gradients of loss with respect to all the weights used in the LSTM cell requires the gradients of all the gates and the candidate state. So, in this section, we will learn how to compute the gradient of the loss function with respect to all of the gates and the candidate state.

Before we begin, let's recollect the following two things:

• The derivative of a sigmoid function is expressed as follows:

• The derivative of a tanh function is expressed as follows:

In the upcoming calculations, we will be ...