Hands-On Python Deep Learning for the Web

Initializing the weight matrix and more

When there are neural networks, there are weights. This is true, right? But before we start to deal with the weights for our RNN, let's see exactly where they are needed.

There are two different weight matrices in the case of an RNN—one for the input neuron (remember that we feed feature vectors only through neurons) and one for the recurrent neuron. A particular state in an RNN is produced using the following two equations:

To understand what each term means in the first equation, refer to the following image (don't worry, we will get to the second equation):

The first pass of the RNN is the letter ...