Feature Engineering Made Easy

How LDA works

LDA works as a dimensionality reduction tool, just like PCA, however instead of calculating the eigenvalues of the covariance matrix of the data as a whole, LDA calculates eigenvalues and eigenvectors of within-class and between-class scatter matrices. Performing LDA can be broken down into five steps:

1. Calculate mean vectors of each class

2. Calculate within-class and between-class scatter matrices

3.  Calculate eigenvalues and eigenvectors for 

4. Keep the top k eigenvectors by ordering them by descending eigenvalues

5. Use the top eigenvectors to project onto the new space

Let's look at an example.