Building Machine Learning Systems with Python - Third Edition

Limitations of PCA and how LDA can help

Being a linear method, PCA has, of course, its limitations when we are faced with data that has nonlinear relationships. We won't go into details here, but it's sufficient to say that there are extensions of PCA, for example, Kernel PCA, that introduce nonlinear transformations so that we can still use the PCA approach.

Another interesting weakness of PCA is when it's being applied to special classification problems. If we replace good = (x1 > 5) | (x2 > 5) with good = x1 > x2 to simulate such a special case, we can quickly see the problem, as can be seen in the following diagram:

Here, the classes are ...