Machine Learning and Big Data with kdb+/q

CHAPTER 16Eigensystem and PCA

In this section, we provide an algorithm to solve the eigensystem, i.e. for any square matrix , we find the eigenvalues and the corresponding eigenvectors. We utilise the eigensystem for the principal component analysis (PCA) decomposition, which is a powerful tool to analyse data.

16.1 THEORY

Matrices and matrix equations are everyday objects we encounter in statistics and data science. Square matrices play a unique role, and it is thus convenient to get more familiar with their properties and be able to deal with them numerically. The covariance matrix is a particular example of the square matrix, which we meet whenever we get our hands on any data set. It is particularly useful to calculate the eigensystem of square matrices as they allow us to make inferences about their properties. When applied to the covariance matrix, we derive the basis for the PCA, which gives insight into the source of variation in the data set.

Let us start with a definition of eigenvalues and eigenvectors, or the eigensystem, for a square matrix. The notion of eigenvectors and eigensystems can be defined for a general linear transformation, but for our purpose, we consider the finite-dimensional linear transformations which can be expressed as square matrices. Let us consider a matrix , which can contain in general complex numbers, and a -dimensional column vector ...