The first RNN layer provided out-of-the-box by TensorFlow.js is the SimpleRNN layer type, which is a layer composed of a SimpleRNNCell neuron. This is an exotic neuron that can feed its output back to its input. The input to such a neuron is a vector of time-dependent values; the activation output of each input value is fed back into the input of the next value, and so on. A dropout factor between 0 and 1 may be specified; this value represents the strength of each echo. A neuron designed in this manner is similar in many ways to an FIR filter.
In fact, this type of RNN architecture is made possible by earlier work in digital signal processing concerning FIR filters. The advantage of this architecture is that the mathematics are ...
