Principal Component Analysis (PCA) is the most famous algorithm for feature extraction. The new feature representation learned by PCA is a linear combination of the original features such that the variance within the original data is preserved as much as possible.
Let's look at this algorithm in action. First, we will consider the dataset we already used the performance prediction. For this problem, the number of features is relatively large; after encoding categorical variables with one-hot-encoding there are more than 1,000 features, and only 5,000 observations. Clearly, 1,000 features is quite a lot for such a small sample size, and this may cause problems when building a machine learning model.
Let's see if ...
