Advanced Natural Language Processing with TensorFlow 2

4

Transfer Learning with BERT

Deep learning models really shine with large amounts of training data. Having enough labeled data is a constant challenge in the field, especially in NLP. A successful approach that has yielded great results in the last couple of years is that of transfer learning. A model is trained in an unsupervised or semi-supervised way on a large corpus and then fine-tuned for a specific application. Such models have shown excellent results. In this chapter, we will build on the IMDb movie review sentiment analysis and use transfer learning to build models using GloVe (Global Vectors for Word Representation) pre-trained embeddings and BERT (Bi-Directional Encoder Representations from Transformers) contextual models. In this chapter, ...