A Sampler of Useful Computational Tools for Applied Geometry, Computer Graphics, and Image Processing

PCA and SVD 57

1

1

Figure 4.10: In general, a transformation may contain rotations and

scales.

A then looks like this:

A = (u

1

u

2

··· u

n

)











σ

1

σ

2

.

.

.

σ

n











(v

1

v

2

··· v

n

)

>

,

i.e., A = UΣV

>

, where Σ is a diagonal matrix with values σ

i

that

are real and nonnegative, and U and V are orthogonal matrices.

This is the singular value decomposition (SVD). The nice thing is

that SVD exists for any matrix of any dimension. In Figure 4.11

we show the SVD for a square matrix.

=

Figure 4.11: Singular value decomposition of a square matrix.

The diagonal values of Σ, σ

1

, σ

2

, . . . , σ

n

are called the singular

values. It is customary to sort them: σ

1

≥ σ

2

≥ . . . ≥ σ

n

≥ 0.

For rectangular ...