Data Analysis: What Can Be Learned From the Past 50 Years

CHAPTER 8

MORE CASE STUDIES

The following case studies concern (i) some non-standard examples of dimension reduction through nonlinear local modeling, and (ii) comparison of point configurations. The latter section was added to give at least one example of quantitative comparisons, and to round off my discussion of the singular value decomposition with a particularly neat application. I have added a brief outline of numerical optimization.

Assume we have a data set with n items and p variables. Any approach to dimension reduction transforms the space of variables in such a way that the leading variables contain most of the information, while the trailing variables contain mostly noise; subsequently the latter are dropped. A standard approach toward selecting a parsimonious model consists in selecting the most informative variables with the help of a criterion such as Mallows’ Cp or Akaike’s AIC. This is not necessarily the best strategy since several variables may measure essentially the same quantity, apart from random fluctuations. Then, a weighted sum of those variables will lead to a better model than any single one of them. The question is how to choose that weighted sum. A general approach is through principal components, that is, through the singular value decomposition. The new, lower-dimensional variable space then is a linear transformation of the old one. For a detailed example see Section 7.2.1.

Even more powerful non-standard approaches proceed through local modeling. ...