Mastering Machine Learning with scikit-learn - Second Edition

Solving OLS for simple linear regression

In this section, we will work through solving OLS for simple linear regression. Recall that simple linear regression is given by the equation y = α + βx and that our goal is to solve for the values of β and α to minimize the cost function. We will solve for β first. To do so, we will calculate the variance of x and the covariance of x and y. Variance is a measure of how far a set of values are spread out. If all the numbers in the set are equal, the variance of the set is zero. A small variance indicates that the numbers are near the mean of the set, while a set containing numbers that are far from the mean and from each other will have a large variance. Variance can be calculated using the following ...