There are a couple of challenges that are listed here that we are trying to solve:
- When we are developing an NLP application, there is one fundamental problem--we know that the machine can't understand our text and we need to convert the text data into a numerical format.
- There are certain ways to convert text data into a numerical format, but we apply some naive techniques, and one of them is one-hot encoding, but the problems with this techniques are given as follows:
- Suppose you have a sentence: I like apple juice. Now, suppose that you apply one-hot encoding for each of the words in the sentence. If you have thousands of sentences in your corpus, then vector dimension is equal to the entire vocabulary of your corpus, and ...