## Description of Least-Squares Estimation

Least-squares estimation determines estimators *b*
_{1} and *b*
_{0} for a function *y*
(*x*) = *b*
_{1} *x* +
*b* _{0} so that
*y* (*x*) is a *best-fit line* through a set of *n* points
(*x* _{0},
*y* _{0} ), . . .,
(*x*_{n} _{ -
1}, *yn* _{ - 1}). A
best-fit line using least-squares estimation minimizes the sum of
squared vertical distances between each point
(*x*_{i} ,
*y*_{i} ),
*i* = 0, . . ., *n* - 1 and a
corresponding point (*x*_{i} ,
*y *(*x*_{i}
)) along *y *(*x*).
This is one way of defining a line so that each point
(*x*_{i} ,
*y*_{i} ) is as close as
possible to it.

Perhaps the most important application of least-squares
estimation is to make inferences about a linear-form relationship
between two variables. Given an independent variable
*x* and a variable *y* that
depends on it, estimators *b*
_{1} and *b*
_{0} allow us to calculate the expected value of
*y* at values of *x* for which
we have not actually observed *y*. This is
particularly meaningful when *x *and
*y* are related by a *statistical
relationship* , which is an inexact relationship. For example, imagine
how the number of new employees hired each month at a consulting firm
is related to the number of hours the firm bills. Generally, as the
firm hires more employees, it will bill more hours. However, there is
not an exact number of hours it bills for a given number of employees.
Contrast this with a *functional relationship*
, which is exact. For example, a functional relationship might be one between the amount of money the firm charges for a project and the time the project requires. ...