Principles of System Identification

12

Introduction to Estimation

The purpose of this chapter is to introduce the reader to the basics of estimation theory. Three

fundamental estimation problems, namely, prediction, filtering and smoothing are reviewed.

Two popularly encountered generic estimation problems, namely, parameter and signal esti-

mation are highlighted. This chapter serves as a stepping stone for the rest of the chapters

devoted to estimation.

12.1 MOTIVATION

Parts I and II laid the theoretical foundations for developing data-driven models for deterministic and

random processes. Different forms of linear time-invariant models that can be possibly built were

studied. We also obtained glimpses of a few tools that can be used for estimating these models. These

tools required the characterization and estimation of a signal’s properties such as auto-covariance

function, spectral density, etc. Having learnt the theoretical definitions and characterizations of the

signal’s properties, the next natural step in identification is to learn how to estimate these signal

properties and model parameters.

From the illustrative examples of the previous chapters, it is clear that the presence of noise

can significantly alter the course of a modeling exercise. It may be required, in many situations,

therefore to filter out the undesirable components of a measurement prior to taking it through the

model estimation exercise. Lastly, but most importantly, is the task of prediction, a fundamental

goal of modeling. Depending on what we wish to predict, the model and the (future) time horizon

of prediction, the mathematics of the model estimation can change significantly.

The full gamut of problems discussed above falls under a single, but the very broad umbrella of

estimation theory. It provides the necessary paraphernalia for estimating the unknown from known

information. The unknown could be a signal, parameter or a state. A secondary, but an important

objective of estimation is to provide (at least under simplistic assumptions) bounds on the errors

incurred in an estimation exercise.

Estimation theory hardly requires any motivating exposition. The problem of estimation arises

in every field of data analysis without any exception. It is only that the forms are different - it could

be in the context of inferring an unknown parameter, or estimating a signal from its measurement,

or predicting the course of a process. All of these problems share a common objective - to discover

or infer the “unobserved truth” from observed data.

The basic concepts of estimation are illustrated by means of an illustrative example. This is the

classic case of estimating a constant (signal) embedded in noise.

12.2 A SIMPLE EXAMPLE: CONSTANT EMBEDDED IN NOISE

Assume that we are interested in knowing the (constant) level x[k] of fluid in a storage tank (no in

and out flow). The level sensor that is being used for this purpose is known to provide an erroneous

measurement y[k]. The true quantity of interest is therefore “hidden” or “unobserved” and has to be

estimated from y[k].

Observation at a single instant in fact does offer an estimate of c. But, as we shall show later, this

is too crude an estimate. Intuitively, using a set of observations {y[0], ··· , y[N −1]} we may obtain

a better estimate (under certain conditions). The estimation problem thus follows.

305