Real-World Natural Language Processing

3 Word and document embeddings

This chapter covers

• What word embeddings are and why they are important
• How the Skip-gram model learns word embeddings and how to implement it
• What GloVe embeddings are and how to use pretrained vectors
• How to use Doc2Vec and fastText to train more advanced embeddings
• How to visualize word embeddings

In chapter 2, I pointed out that neural networks can deal only with numbers, whereas almost everything in natural language is discrete (i.e., separate concepts). To use neural networks in your NLP application, you need to convert linguistic units to numbers, such as vectors. For example, if you wish to build a sentiment analyzer, you need to convert the input sentence (sequence of words) into a sequence of vectors. ...