R Deep Learning Projects

word2vec

The word2vec algorithm (or, rather, family of algorithms) takes a text corpus as input and produces the word vectors as output. It first constructs a vocabulary from the training text data and then learns vector representation of words. Then we use those vectors as features for machine learning algorithms.

Word vectors are able to catch some intuitive regularities in the language, for instance:

vector('Paris') - vector('France') + vector('Italy')

Results in a vector that is very close to:

vector('Rome'),

And,

vector('king') - vector('man') + vector('woman') 

Is close to:

vector('queen').

What does word2vec do behind the scenes? word2vec arrives at word vectors by training a neural network to predict:

• A word in the center from its ...