Getting ready

GloVe aims to achieve two goals:

  • Creating word vectors that capture meaning in vector space
  • Taking advantage of global count statistics instead of only local information

GloVe learns word vectors by looking at the cooccurrence matrix of words and optimizing for a loss function. The working details of the GloVe algorithm can be understood from the following example:

Let's consider a scenario where there are two sentences, as follows:

Sentences

This is test

This is also a

 

Let's try to build a word-cooccurrence matrix. There is a total of five unique words within our toy dataset of sentences, and from there the word-cooccurrence matrix looks as follows:

In the preceding table, the words this and is occur together ...

Get Neural Networks with Keras Cookbook now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.