Hands-On Unsupervised Learning with Python

Adding more robustness to heteroscedastic noise with factor analysis

One of the main problems with standard PCA is the intrinsic weakness of such a model in terms of heteroscedastic noise. If you are not familiar with this terminology, it will be helpful to introduce two definitions. A multivariate decorrelated noise term is characterized by a diagonal covariance matrix, C, which can have two different configurations, as follows:

• C = diag(σ², σ², ..., σ²): In this case, the noise is defined as homoscedastic (all of the components have the same variance).
• C = diag(σ₁², σ₂², ..., σ_n²), with σ₁² ≠ σ₂² ≠ ... ≠ σ_n²: In this case, the noise is defined as heteroscedastic (every component has its own variance).

It's possible to prove that, when ...