Nonlinear Filters

12Nonparametric Bayesian Models

12.1 Introduction

In Bayesian models, any form of uncertainty is viewed as randomness, and model parameters are usually assumed to be random variables [248]. Parametric and nonparametric Bayesian models are defined on finite‐ and infinite‐dimensional parameter spaces, respectively. While finite‐dimensional probability models are described using densities, infinite‐dimensional probability models are described based on concepts of measure theory. For a general type of observations known as exchangeable sequences, existence of a randomly distributed parameter is a mathematical consequence of the characteristics of the collected data rather than being a modeling assumption [249]. Nonparametric Bayesian models can be derived by starting from a parametric model and then taking the infinite limit [250]. Existence of noncomputable conditional distributions as well as limitations of using computable probability distributions for describing and computing conditional distributions would rule out the possibility of deriving generic probabilistic inference algorithms (even inefficient ones). However, the presence of additional structure such as exchangeability, which is common in Bayesian hierarchical modeling, paves the way for posterior inference [251].

12.2 Parametric vs Nonparametric Models

A parametric model uses a finite set of parameters to capture what can be known from a dataset, which is relevant for a prediction task. Mathematically speaking, ...