Mastering Machine Learning with scikit-learn - Second Edition

Variance, covariance, and covariance matrices

There are several terms that we must define before discussing how PCA works. Recall that variance is a measure of how spread out a set of values is. Variance is calculated as the average of the squared differences of the values and the mean of the values, per the following equation:

Covariance is a measure of how much two variables change together; it is a measure of the strength of the correlation between two sets of variables. If the covariance of two variables is zero, the variables are uncorrelated. Note that uncorrelated variables are not necessarily independent, as correlation is only a measure ...