Neural Networks with Keras Cookbook

Intuition of RNN architecture

RNN is useful when we want to predict the next event given a sequence of events.

An example of that could be to predict the word that comes after This is an _____.

Let's say, in reality, the sentence is This is an example.

Traditional text-mining techniques would solve the problem in the following way:

1. Encode each word while having an additional index for potential new words:

This: {1,0,0,0}is: {0,1,0,0}an: {0,0,1,0}

2. Encode the phrase This is an:

This is an: {1,1,1,0}

3. Create the training dataset:

Input --> {1,1,1,0}Output --> {0,0,0,1}

4. Build a model with input and output

One of the major drawbacks of the model is that the input representation does not change in the input sentence; it is either this is ...