Hands-On Transfer Learning with Python

Shortcomings of BoW models

With the word count-based BoW model, we lose additional information such as the semantics, structure, sequence, and context around nearby words in each text document. Words with similar meaning are treated differently in BoW. Other models of text are latent semantic indexing (LSI), where the documents are represented in a low dimensional (k << vocabulary size)—hidden topic space. In LSI, the constituent words in the document can also be represented as a k dimensional dense vector. It's observed that, in LSI models, words with similar semantics have close representations. Also, this dense representation of words is the first step for applying deep learning models to text and is called word embedding. Neural network-based ...