Skip to Main Content
Getting Started with Google BERT
book

Getting Started with Google BERT

by Sudharsan Ravichandiran
January 2021
Beginner to intermediate content levelBeginner to intermediate
352 pages
10h 17m
English
Packt Publishing
Content preview from Getting Started with Google BERT
BERT Variants I - ALBERT, RoBERTa, ELECTRA, and SpanBERT

In this chapter, we will understand different variants of BERT, such as ALBERT, RoBERTa, ELECTRA, and SpanBERT. We will start with understanding how ALBERT works. ALBERT is basically A Lite version of BERT model. The ALBERT model includes few architectural changes to the BERT to minimize the training time. We will cover how ALBERT works and how it differs from BERT in detail. 

Moving on, we will learn about the RoBERTa model, which stands for a Robustly Optimized BERT pre-training Approach. RoBERTa is one of the most popular variants of the BERT and it is used in many state-of-the-art systems. RoBERTa works similar to BERT but with a few changes in the pre-training steps. We will explore ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Clean Code Fundamentals

Clean Code Fundamentals

Robert C. Martin

Publisher Resources

ISBN: 9781838821593Supplemental Content