Hands-On Natural Language Processing with Python

Input module

The input module does a word embedding lookup for all the words in each input fact, and then builds a single embedding for each fact by summing across the temporal direction, that is, summing the word embeddings for each word in a fact:

    def _input_module(self, facts):        with tf.variable_scope("InputModule"):            facts_emb = tf.nn.embedding_lookup(self.word_emb_matrix,                                                facts)            return tf.reduce_sum(facts_emb, 2)