Deep Learning with TensorFlow

PTB model

The PTB model is implemented through the PTBModel class that you can find in the ptb_word_lm.py file. Here we analyze the basic pseudo code.

The network model consists of a BasicLSTMCell cell:

    lstm = rnn_cell.BasicLSTMCell(lstm_size)

The memory state of the network is initialized with a vector of zeros; the data is processed in a mini batch set of size batch_size:

state = tf.zeros([batch_size, lstm.state_size])

For each analyzed word, the probabilities for the continuations of the sentence are computed as follows:

probabilities = []for current_batch_of_words in words_in_dataset:

The value of state is updated after processing each batch of words:

    output, state = lstm(current_batch_of_words, state) 

The LSTM output is then ...