R Deep Learning Essentials - Second Edition

Word vectors

Instead of representing our text data as a bag of words, deep learning represents them as word vectors or embeddings. A vector/embedding is nothing more than a series of numbers that represent a word. You may have already heard of popular word vectors such as Word2Vec and GloVe. The Word2vec model was invented by Google (Mikolov, Tomas, et al. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)). In their paper, they provide examples of how these word vectors have somewhat mysterious and magical properties. If you take the vector of the word "King", subtract the vector of the word "Man", add the vector of the word "Man", then you get a value close to the vector of the word "Queen" ...