Neural Network Programming with TensorFlow

Defining Twinet

Twisting RNNs (Twinet) use two parallel branches. Each branch is composed of a recurrent network layer, a nonlinear perceptron layer, and a reversed recurrent network layer. Branches are twisted: the order of the layers is reversed in the second branch. The output of all the recurrent layers is collected toward the end.

To recap, a recurrent neural network (RNN) takes a sequence of input vectors x1..T, and recurrently computes hidden states (also called output labels):

h_t = σ(U · x_t + W · ht−1)

where,

• t is 1..T
• x_t is the external signal
• W are the weights
• h_t-1 is the hidden layer weights for time step t-1
• h_t weights being calculated for time step t
• U is tanh layer which helps in creating weights for time step t

σ(·) is a ...