Hands-on Question Answering Systems with BERT: Applications in Neural Networks and Natural Language Processing

© Navin Sabharwal, Amit Agrawal 2021

N. Sabharwal, A. AgrawalHands-on Question Answering Systems with BERThttps://doi.org/10.1007/978-1-4842-6664-9_4

4. BERT Algorithms Explained

Navin Sabharwal¹   and Amit Agrawal²

(1)

New Delhi, Delhi, India

(2)

Mathura, India

 

This chapter takes a deep dive into the BERT algorithm for sentence embedding along with various training strategies, including MLM and NSP. We will also see an implementation of a text classification system using BERT.

How Does BERT Work?

BERT makes use of a transformer to learn contextual relations between words in a text. A transformer has two mechanisms—an encoder and a decoder—but BERT only requires the encoder mechanism. BERT uses a bidirectional approach and reads the text input sequentially, ...