TensorFlow 1.x Deep Learning Cookbook

How to do it...

We proceed with the recipe as follows:

1. Define an attention mechanism by using the library tf.contrib.seq2seq.LuongAttention, which implements the attention model defined in Effective Approaches to Attention-based Neural Machine Translation by Minh-Thang Luong, Hieu Pham, and Christopher D. Manning (2015):

# attention_states: [batch_size, max_time, num_units] attention_states = tf.transpose(encoder_outputs, [1, 0, 2])  # Create an attention mechanism attention_mechanism = tf.contrib.seq2seq.LuongAttention( num_units, attention_states, memory_sequence_length=source_sequence_length)

2. Use the defined attention mechanism as a wrapper around the decoder cell by means of an attention wrapper:

decoder_cell = tf.contrib.seq2seq.AttentionWrapper( ...