6.5 Regularization parameter selection
We have to this point avoided the question of how to choose the parameter that appears in the penalized least-squares criterion. There are several data adaptive techniques that are typically used for this purpose. We will describe three of them that are appropriate for the scenario that was examinedin the previous section: namely, where and for finite dimensional with basis . As before the squared norm for is taken to be
We begin with ordinary cross validation. Let be the minimizer of
Ordinary cross validation then picks to minimize ...