Optimisation in Signal and Image Processing

Chapter 4

Information Criteria: Examples of Applications in Signal and Image Processing

4.1. Introduction and context

In this chapter we will focus on the sequence of N observations x^N = (x_l, ..., x_N) on a stationary and random process consisting of a family of random variables X = {X_n}_nZ distributed according to the same unknown law θ. A model θ_k based on k free parameters will represent this process X. Determining the optimal estimation _k of θ_k in the maximum likelihood (ML) sense enables us to find _k which maximizes f(x^N |θ_k) where f represents the conditional density probability of the observations x^N when choosing the model θ_k. Finding _k which will minimize L(θ_k) = −log f(x^N |θ_k) and therefore has the same effect.

Even though this criterion of estimation is expressed by a fixed number k, it might be tempting to use this criterion to carry out a simultaneous estimation of the model’s parameters and its number of free parameters, which in a written form can be expressed as follows: ...