Python Natural Language Processing

Skip-gram

The skip-gram (SG) model reverses the usage of target words and context words. Here, the target word is given as input to the input layer in the form of a one-hot encoded vector. The hidden layer remains the same. The output layer of the neural network is repeated multiple times to generate the chosen number of context words.

Let's take an example of the words cat and tree as context words and the word climbed as a target word. The input vector in the SG model will be the one-hot encoded word vector of the word climbed [0 0 0 1 0 0 0 0 ]t and this time, our output vectors should be vectors for the word cat and the word tree. So, the output vector should be [ 0 1 0 0 0 0 0 0] for cat and [0 0 0 0 0 0 0 1] for tree. Refer to the structure ...