Hands-On Deep Learning Algorithms with Python

Updating hidden state

We just learned how the information in the cell state will be updated. Now, we will see how the information in the hidden state will be updated. We learned that the hidden state, , is used for computing the output, but how can we compute the output?

We know that the output gate is responsible for deciding what information should be taken from the cell state to give as output. Thus, multiplying and tanh (to squash between -1 and +1) of cell state, , gives us the output.

Thus, the hidden state, , is expressed as follows: ...