O'Reilly logo

Machine Learning with Spark - Second Edition by Nick Pentreath, Manpreet Singh Ghotra, Rajdeep Dua

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Skip-gram model

The training objective of the skip-gram model is to find word representations useful for predicting the surrounding words in a document or a sentence. Given a sequence of words w1, w2, w3, . . , wT, skip-gram model maximizes the average log probability shown as following:

c is the size of the training context (which can be a function of the center word wt). Larger c results in more training examples leading to a higher accuracy, at the expense of the training time. The basic skip-gram formulation defines p(wt+j |wt) using the softmax function:

vw, v' and, w are the input and output vector representations of w, and W is the ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required