Mastering Algorithms with C

Description of Least-Squares Estimation

Least-squares estimation determines estimators b ₁ and b ₀ for a function y (x) = b ₁ x + b ₀ so that y (x) is a best-fit line through a set of n points (x ₀, y ₀ ), . . ., (x_{n - 1}, yn _{- 1}). A best-fit line using least-squares estimation minimizes the sum of squared vertical distances between each point (x_i , y_i ), i = 0, . . ., n - 1 and a corresponding point (x_i , y (x_i )) along y (x). This is one way of defining a line so that each point (x_i , y_i ) is as close as possible to it.

Perhaps the most important application of least-squares estimation is to make inferences about a linear-form relationship between two variables. Given an independent variable x and a variable y that depends on it, estimators b ₁ and b ₀ allow us to calculate the expected value of y at values of x for which we have not actually observed y. This is particularly meaningful when x and y are related by a statistical relationship , which is an inexact relationship. For example, imagine how the number of new employees hired each month at a consulting firm is related to the number of hours the firm bills. Generally, as the firm hires more employees, it will bill more hours. However, there is not an exact number of hours it bills for a given number of employees. Contrast this with a functional relationship , which is exact. For example, a functional relationship might be one between the amount of money the firm charges for a project and the time the project requires. ...