5 State-of-the-art in deep learning: Transformers
This chapter covers
- Representing text in numerical format for machine learning models
- Building a Transformer model using the Keras sub-classing API
We have seen many different deep learning models so far, namely fully connected networks, convolutional neural networks, and recurrent neural networks. We used a fully connected network to reconstruct corrupted images, a convolutional neural network to classify vehicles from other images, and finally an RNN to predict future CO2 concentration values. In this chapter we are going to talk about a new type of model known as the Transformer.
Transformers are the latest generation of deep networks to emerge. Vaswani et al., in their paper “Attention Is ...
Get TensorFlow in Action now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
