Hands-On Deep Learning Algorithms with Python

Understanding skip-gram model

Now, let's look at another interesting type of the word2vec model, called skip-gram. Skip-gram is just the reverse of the CBOW model,. That is in a skip-gram model, we try to predict the context words given the target word as an input. As shown in the following figure, we can notice that we have the target word as rises and we need to predict the context words the, sun, in, and the:

Similar to the CBOW model, we use the window size to determine how many context words we need to predict. The architecture of the skip-gram model is shown in the following figure.

As we can see that it takes the single target word ...