GloVe aims to achieve two goals:
- Creating word vectors that capture meaning in vector space
- Taking advantage of global count statistics instead of only local information
GloVe learns word vectors by looking at the cooccurrence matrix of words and optimizing for a loss function. The working details of the GloVe algorithm can be understood from the following example:
Let's consider a scenario where there are two sentences, as follows:
|
Sentences |
|
This is test |
|
This is also a |
Let's try to build a word-cooccurrence matrix. There is a total of five unique words within our toy dataset of sentences, and from there the word-cooccurrence matrix looks as follows:
In the preceding table, the words this and is occur together ...
