Deep Learning By Example

Feature selection

The number of explanatory features (input variables) of a sample can be enormous wherein you get x_i=(x_i¹, x_i², x_i³, ... , x_i^d) as a training sample (observation/example) and d is very large. An example of this can be a document classification task3, where you get 10,000 different words and the input variables will be the number of occurrences of different words.

This enormous number of input variables can be problematic and sometimes a curse because we have many input variables and few training samples to help us in the learning procedure. To avoid this curse of having an enormous number of input variables (curse of dimensionality), data scientists use dimensionality reduction techniques in order to select a subset from ...