Principles of System Identification

16

Estimation of Signal Properties

This chapter serves as a useful reference for estimators of the signal’s properties, both sta-

tistical and deterministic. In the first part of this chapter, we shall study expressions for esti-

mating time-domain properties. The second part of this chapter is devoted to the estimation

of frequency-domain characteristics, particularly, the spectral densities. The objective is not

merely to provide expressions for the estimators but also to highlight their properties so that

the user is aware of the appropriateness of a particular estimator for a given application.

16.1 INTRODUCTION

Preceding chapters have highlighted the need for estimating the signal properties and also discussed

in detail the methods to obtain the same. Estimation of the properties of signals, be it output or input,

is important at different stages of identification. In Chapter 2 we observed that the first step towards

building a (linear) data-driven model is the construction of deviation quantities. A generalization of

this step is the removal of trends. In all estimation exercises, an important post-estimation step is the

computation of standard errors, which requires the know-how of computing variance of observation

noise. The Fisher’s information metric gives us the extent of information available in a data given

a theoretical model. In practice, we need an empirical version of the same. On the other side of the

story is the need to compute spectral densities for the estimation of frequency response functions

and input-output delays.

This chapter is solely devoted to address the aforementioned needs. First we present estimators of

key statistical properties, namely, mean, variance and correlation (functions). Subsequently two em-

pirical methods for computing Fisher’s information are reviewed. Estimation of frequency-domain

properties, namely the auto- and the cross-power spectral densities constitute the second-half of this

chapter. Suitable illustrations using the support software are provided to assist the reader in taking

the estimators from theory to practice.

A cautionary note is in place here. It is a good practice to distinguish theoretical definitions from

the expressions (formulae) used to obtain the corresponding estimates. For instance, quite often the

sample mean is confused with the true definition of mean of a signal. In principle they are different

and coincide only for a class of signals or under special conditions. For any property of a signal,

there exists only one theoretical definition, but many estimators. The presentation to follow does not

recommend a particular estimator, but rather discusses the properties and highlights certain known

aspects. It is the user’s call to make the “right” choice of estimator, which is largely governed by its

properties, as discussed in Chapter 13 and any other application-specific criteria (e.g., ease of online

implementation).

16.2 ESTIMATION OF MEAN AND VARIANCE

We examine the estimators of two fundamental properties of a signal, namely, mean and variance.

The theoretical definitions for these quantities for stochastic signals were provided in (7.8) and

(7.14), respectively.

419

420 Principles of System Identification: Theory and Practice

16.2.1 ESTIMATORS OF MEAN

Several estimators of mean exist, of which we discuss two widely used estimators. In the dis-

cussions to follow, the vector of N observations are denoted by a boldfaced vector, for e.g.,

y = {y[0], y[1], y[N − 1]}.

Sample mean

The sample mean, encountered in several discussions earlier, is given by:

¯y =

N −1

X

k=0

α

k

y[k] where α

k

=

1

N

, ∀ k (16.1)

Properties of sample mean

i. The sample mean is the MoM and OLS estimator of the mean. It is also the MLE with y[k] ∼

GWN(µ, σ

2

) assumption.

ii. Unbiasedness: The estimator is unbiased as long as the generating process is stationary.

iii. Variance: Under the stationarity assumption for {y[k]} its variability is given by:

σ

2

¯y

=

1

N

N −1

X

l=−(N −1)

1 −

|l |

N

!

σ

yy

[l] (16.2)

where σ

yy

[l] is the ACVF of {y[k]}.

iv. Consistency: The estimator converges in the mean square sense (and consequently in probabil-

ity). In other words, it is consistent. The stationarity requirement on y[k] is implicitly understood.

v. Efficiency: The sample mean is the most efficient estimator when {y[k]} is Gaussian white. For

correlated processes, a weighted least squares estimate or MLE with the appropriate correlation

function should be used.

vi. Distribution: The CLT establishes the asymptotic distribution of ¯y under fairly general condi-

tions. A more precise statement is, however, necessary when the process is correlated.

Theorem 16.1

If {y[k]} has a linear stationary representation

y[k] = µ

y

+

∞

X

n=−∞

h[n]w[k − n] under w[k] ∼ i.i.d.(0, σ

2

w

) (16.3)

then,

¯y ∼ AsN(µ,

λ

N

) where λ =

∞

X

l=−∞

1 −

|l |

N

!

σ

yy

[l] (16.4)

Proof. See Shumway and Stoffer (2006). 