Digital Signal Processing with Kernel Methods

8Advances in Kernel Regression and Function Approximation

8.1 Introduction

Kernel methods constitute a proper framework to tackle regression problems that encompass fitting and regularization. In this chapter we will pay attention to two particularly interesting ways of treating the regression problem: based on discriminative kernel regression, and based on generative Bayesian nonparametric regression. Both families have found wide application in signal processing in the last decade (Pérez‐Cruz et al., 2013 ; Rojo‐Álvarez et al., 2014).

The chapter is divided into two main sections. In Section 8.2, we will depart from the standard SVR method (Smola and Schölkopf, 2004) extensively used in the previous chapters. The method allows many alternative formulations to deal with the specificities of the DSP problems; for example, dealing with multiple outputs, unlabeled data, and signal‐dependent noise sources and heteroscedastic models. Recent approaches to treat such problems will be introduced and experimentally evaluated. In all of them, the important role of both the loss and the regularizer will appear, as well as the design of the kernel function. In Section 8.3, as an alternative to this treatment based on the SVR algorithm, we will review two instantiations of the field of Bayesian nonparametrics: relevance vector machines (RVMs) (Tipping, 2000) and Gaussian processes (Rasmussen and Williams, 2006). We study the important issue of model selection and hyperparameters search ...