Machine Learning

Chapter 19

Dimensionality Reduction and Latent Variables Modeling

Abstract

This chapter deals with latent variables modeling and dimensionality reduction techniques. It starts with the more classical principle components analysis (PCA) method. Its various properties are analyzed and its interpretation as a low-rank matrix factorization is emphasized. Then, the canonical correlation analysis (CCA) and its relatives, such as partial least-squares (PLS) are introduced. Independent component analysis (ICA) is reviewed and the cocktail party problem is presented. Dictionary learning, as a matrix factorization approach, is defined and the k-SVD algorithm is considered. The probabilistic approach to latent variables modeling is reviewed, starting ...