Advanced Deep Learning with Python

RNN implementation and training

In the preceding section, we briefly discussed what RNNs are and what problems they can solve. Let's dive into the details of an RNN and how to train it with a very simple toy example: counting ones in a sequence.

In this problem, we will teach a basic RNN how to count the number of ones in the input and then output the result at the end of the sequence. This is an example of a many-to-one relationship, which we defined in the previous section.

We'll implement this example with Python (no DL libraries) and NumPy. An example of the input and output is as follows:

In: (0, 0, 0, 0, 1, 0, 1, 0, 1, 0) Out: 3

The RNN we'll use is illustrated in the following diagram:

Basic RNN for counting ones in the input

The ...