Mastering pandas - Second Edition

Deviation and variance

A fundamental idea in the discussion of variability is the concept of deviation. Simply put, a deviation measure tells us how far away a given value is from the mean of the distribution—that is, .

To find the deviation of a set of values, we define the variance as the sum of the squared deviations and normalize it by dividing it by the size of the dataset. This is referred to as the variance. We need to use the sum of the squared deviations. By taking the sum of the deviations around the mean results in 0, since the negative and positive deviations cancel each other out. The sum of the squared deviations is defined as ...