Machine Learning Algorithms - Second Edition

Dimensionality reduction

This step is not always mandatory, but, in many cases, it can be a good solution to avoid memory leaks or long computational times. When the dataset has many features, the probability of some hidden correlation is relatively high. For example, the final price of a product is directly influenced by the price of all materials, and, if we remove one secondary element, the value changes slightly (generally speaking, we can say that the total variance is almost preserved). If you remember how Principal Component Analysis (PCA) works, you know that this process decorrelates the input data too. Therefore, it's useful to check whether a PCA or a Kernel PCA (for non-linear datasets) can remove some components while keeping ...