Python Natural Language Processing

Continuous bag of words

In the continuous bag of words (CBOW) algorithm, context is represented by multiple words for given target words. Just recall our example that we stated in an earlier section, where our context word was cat and our target word was climbed. For example, we can use cat as well as tree as context words to predict the word climbed as the target word. In this case, we need to change the architecture of the neural network, especially the input layer. Now, our input layer may not represent the single-word one-hot encode vector, but we need to put another input layer that represents the word tree.

If you increase the context words, then you need to put an additional input layer to represent each of the words, and all these ...