Deep Learning Quick Reference

Network architecture

Our network will use two Keras LSTM layers, each with 100 LSTM units:

inputs = Input(batch_shape=(batch_shape, sequence_length,                input_dim), name="input")lstm1 = LSTM(100, activation='tanh', return_sequences=True,              stateful=True, name='lstm1')(inputs)lstm2 = LSTM(100, activation='tanh', return_sequences=False,              stateful=True, name='lstm2')(lstm1)output = Dense(1, activation='tanh', name='output')(lstm2)

Pay special attention to the return_sequences argument. When connecting two LSTM layers, you need the previous LSTM layer to output predictions for each time step in the sequence so that the input for the next LSTM layer is three-dimensional. Our Dense layer, however, only needs a two-dimensional output in order to predict ...