Neural Network Projects with Python

What's inside an RNN?

Let's now take a closer look at what goes on inside each layer of an RNN. The following diagram depicts the mathematical function inside each layer of an RNN:

The mathematical function of an RNN is simple. Each layer t within an RNN has two inputs:

• The input from the time step t
• The hidden state passed from the previous layer t-1

Each layer in an RNN simply sums up the two inputs and applies a tanh function to the sum. It then outputs the result, to be passed as a hidden state to the next layer. It's that simple! More formally, the output hidden state of layer t is this:

But what exactly is the tanh function? The ...