Python Deep Learning - Second Edition

RNN implementation and training

In the preceding section, Recurrent neural networks, we briefly discussed what RNNs are and what problems they can solve. Let's dive into the details of an RNN and how to train it with a very simple toy example: counting ones in a sequence.

In this problem, we will teach a basic RNN how to count the number of ones in the input, and then output the result at the end of the sequence. This is an example of a "many-to-one" relationship, which we defined in the previous section.

We'll implement this example with Python (no DL libraries) and NumPy. An example of input and output is as follows:

In: (0, 0, 0, 0, 1, 0, 1, 0, 1, 0) Out: 3

The RNN we'll use is illustrated in the following diagram:

Basic RNN for counting ...