Chapter 11

The Least Squares Method

In this chapter, we are going to present a series of techniques based on minimizing mean square criteria to solve linear problems. But first we are going to state a fundamental theorem, called the projection theorem. It was mentioned more or less explicitly in the affine trend suppression problem, or when we estimated the amplitudes of a harmonic signal's components. We will see that it has major applications both in a deterministic or random context.

11.1    The projection theorem

The projection theorem is presented in mathematical form. However, readers that are not used to this formalism should not be worried, since the result expressed by relation 11.1 is quite intuitive, as it is shown in Figure 11.1.

Definition 11.1 (Hilbert space) Let Image be a vector space with a dot product (x, y) for any two of its elements:

  • The norm of an element x of Image is the positive number defined by || x || = Image.
  • x and y are said to be orthogonal, which is denoted by xy, if (x, y) = 0.
  • The distance between two elements x and y of Image is the positive number defined by d(x, ...

Get Digital Signal and Image Processing Using MATLAB now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.