Scala Machine Learning Projects

Contextual information and the architecture of RNNs

Human beings don't start thinking from scratch; the human mind has so-called persistence of memory, the ability to associate the past with recent information. Traditional neural networks, instead, ignore past events. For example, in a movie scenes classifier, it's not possible for a neural network to use a past scene to classify current ones. RNNs were developed to try to solve this problem:

Figure 1: RNNs have loops

In contrast to conventional neural networks, RNNs are networks with a loop that allows the information to be persistent (Figure 1). In a neural network say, A: at some time ...