Hands-On Machine Learning with C++

GloVe

Notice that the Word2Vec algorithm only takes the local context of words into account. It doesn't use global statistics throughout the training corpus. For example, the words the and cat will often be used together, but Word2Vec is unable to determine whether the word the is a standard article or whether the words the and cat have a strong implicit connection. This problem leads to the idea of using global statistics.

For example, latent semantic analysis (LSA) calculates the embeddings for words by factorizing the term-document matrix using singular decomposition. However, even though this method takes advantage of global information, the obtained vectors do not show the same generalization properties as those obtained using Word2Vec. ...