Deep Learning Essentials

The word windows

Remember that from the section on the idea of word embedding, we know that a word can be represented by its context, or more specifically, its so words. Therefore, we can use a window to determine how many surrounding words (before and after) we’d like to learn together with the target word in the center, as shown in the following figure:

In this case, the window size is 2. In order to learn the target word sits, nearby words up to two words away from the target word will be included to generate the training pairs. Then we slide the window along structured texts.