Hands-On Natural Language Processing with Python

Encoder-decoder architecture

As the name encoder-decoder suggests, this architecture consists of an encoder and a decoder component. This is illustrated in the following diagram. The function of the encoder is to take an input text sequence and convert it into a dense vector representation, also known as a thought vector or context vector. The thought vector, in essence, is an internal representation that captures the context and meaning of the whole input text. The decoder takes the dense vector representation of the original, complete text and generates the output summary, one word at a time: