Practical Linear Algebra for Data Science

Chapter 13. Eigendecomposition

Eigendecomposition is a pearl of linear algebra. What is a pearl? Let me quote directly from the book 20,000 Leagues Under the Sea:

For poets, a pearl is a tear from the sea; for Orientals, it’s a drop of solidified dew; for the ladies it’s a jewel they can wear on their fingers, necks, and ears that’s oblong in shape, glassy in luster, and formed from mother-of-pearl; for chemists, it’s a mixture of calcium phosphate and calcium carbonate with a little gelatin protein; and finally, for naturalists, it’s a simple festering secretion from the organ that produces mother-of-pearl in certain bivalves.

Jules Verne

The point is that the same object can be seen in different ways depending on its use. So it is with eigendecomposition: eigendecomposition has a geometric interpretation (axes of rotational invariance), a statistical interpetation (directions of maximal covariance), a dynamical-systems interpretation (stable system states), a graph-theoretic interpretation (the impact of a node on its network), a financial-market interpretation (identifying stocks that covary), and many more.

Eigendecomposition (and the SVD, which, as you’ll learn in the next chapter, is closely related to eigendecomposition) is among the most important contributions of linear algebra to data science. The purpose of this chapter is to provide you an intuitive understanding of eigenvalues and eigenvectors—​the results of eigendecomposition of a matrix. Along the way, you’ll ...