Python Natural Language Processing

Curse of dimensionality

Let's understand what I mean by the curse of dimensionality because this concept will help us understand why we need feature selection techniques. The curse of dimensionality says that, as the number of features or dimensions grows, which means adding new features to our machine learning algorithm, then the amount of data that we need to generalize accurately grows exponentially. Let's see with an example.

Suppose you have a line, one-dimensional feature space, and we put five points on that line. You can see that each point takes some space on this line. Each point takes one-fifth of the space on the line. Refer to Figure 8.64:

Figure 8.64: A one-dimensional features space with five data points

If you have two-dimensional ...