Intuition of RNN architecture

RNN is useful when we want to predict the next event given a sequence of events.

An example of that could be to predict the word that comes after This is an _____.

Let's say, in reality, the sentence is This is an example.

Traditional text-mining techniques would solve the problem in the following way:

  1. Encode each word while having an additional index for potential new words:
This: {1,0,0,0}is: {0,1,0,0}an: {0,0,1,0}
  1. Encode the phrase This is an:
This is an: {1,1,1,0}
  1. Create the training dataset:
Input --> {1,1,1,0}Output --> {0,0,0,1}
  1. Build a model with input and output

One of the major drawbacks of the model is that the input representation does not change in the input sentence; it is either this is ...

Get Neural Networks with Keras Cookbook now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.