Mastering Machine Learning with Spark 2.x

The skip-gram model

In the previous model, we used a window of words before and after the focus word to predict the focus word. The skip-gram model takes a similar approach but reverses the architecture of the neural network. That is, we are going to start with the focus word as our input into our network and then try to predict the surrounding contextual words using a single hidden layer:

As you can see, the skip-gram model is the exact opposite of the CBOW model. The training goal of the network is to minimize the summed prediction error across all the context words in the output layer, which, in our example, is an input of ideas and an ...