O'Reilly logo

Hands-On Natural Language Processing with Python by Rajalingappaa Shanmugamani, Rajesh Arumugam

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

A comparison of skip-gram and CBOW model architectures

The following diagram shows a comparison of the CBOW and skip-gram model architectures:

This diagram shows how skip-gram learns to predict the target word by using the words nearby. On the other hand, CBOW learns to predict the target word based on the words present in its context, which is represented as a bag-of-words, present in a fixed-size window that is around the target word.

In general, the skip-gram approach tends to produce better word representations when the datasets are larger. Hence, we will concentrate on building a skip-gram model for the rest of this chapter. We will also ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required