Hands-On Machine Learning with C++

Singular value decomposition

Singular value decomposition (SVD) is an important method that's used to analyze data. The resulting matrix decomposition has a meaningful interpretation from a machine learning point of view. It can also be used to calculate PCA. SVD is rather slow. Therefore, when the matrices are too large, randomized algorithms are used. However, the SVD calculation is computationally more efficient than the calculation for the covariance matrix and its eigenvalues in the original PCA approach. Therefore, PCA is often implemented in terms of SVD. Let's take a look.

The essence of SVD is pure—any matrix (real or complex) is represented as a product of three matrices:

Here, is a unitary matrix of order and is a matrix ...