Hands-On Natural Language Processing with Python

TensorBoard visualization

We will now take a brief look at the graphs and the loss function, with TensorBoard. First, we will look at the graph output from TensorBoard:

Graph output on TensorBoard

We can clearly see that the encoding layer and decoding layer form the main components of the model, with the Bahdanau attention mechanism as another input in the decoding layer. The input embeddings are first fed into the encoding layer, whose output is fed to the attention mechanism, as well as the decoding layer. Finally, the decoding layer provides the predictions as output. The optimizer takes in the decoding layer output, the attention mechanism ...