Introduction to Computational Linear Algebra

Chapter 4

Orthogonal Factorizations and Linear

Least Squares Problems

The purpose of linear least squares approaches is mainly to approximate so-

lutions of overdetermined systems, i.e., sets of equations in which there are

more equations than unknowns.

4.1 Formulation of Least Squares Problems: Regression

Analysis

Specifically, consider a matrix A ∈ R

m×n

, and let y ∈ R

m

. The least

squares solution w ∈ R

n

of the linear system Aw“ = ”y is defined as:

Find w ∈ R

n

, such that ky − Awk = min

z∈R

n

ky − Azk, (4.1)

where for x ∈ R

n

, kxk = kxk

2

=

√

x

T

x is the l

2

vector norm. The existence

and uniqueness of such a solution will be discussed in Sections 4.2 and 4.3.

We first ...