Skip to Main Content
Getting Started with Google BERT
book

Getting Started with Google BERT

by Sudharsan Ravichandiran
January 2021
Beginner to intermediate content levelBeginner to intermediate
352 pages
10h 17m
English
Packt Publishing
Content preview from Getting Started with Google BERT
Understanding the BERT Model

In this chapter, we will get started with one of the most popularly used state-of-the-art text embedding models called BERT. BERT has revolutionized the world of NLP by providing state-of-the-art results on many NLP tasks. We will begin the chapter by understanding what BERT is and how it differs from the other embedding models. We will then look into the working of BERT and its configuration in detail.

Moving on, we will learn how the BERT model is pre-trained using two tasks, called masked language modeling and next sentence prediction, in detail. We will then look into the pre-training procedure of BERT. At the end of the chapter, we will learn about several interesting subword tokenization algorithms, including ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Clean Code Fundamentals

Clean Code Fundamentals

Robert C. Martin

Publisher Resources

ISBN: 9781838821593Supplemental Content