Deep Learning for Natural Language Processing

10 Applications of Transformers: Hands-on with BERT

This chapter covers

• Creating a BERT layer for importing existing BERT models
• Training BERT on data
• Fine-tuning BERT
• Extracting embeddings from BERT and inspecting them

This chapter addresses the practicalities of working with the BERT Transformer in your implementations. We will not implement BERT ourselves—that would be a daunting job and unnecessary since BERT has been implemented efficiently in various frameworks, including Keras. But we will get close to the inner workings of BERT code. We saw in chapter 9 that BERT has been reported to improve NLP applications significantly. While we do not carry out an extensive comparison in this chapter, you are encouraged to revisit the applications ...