Principles of System Identification

21

Identification of Parametric

Input-Output Models

This chapter presents methods for identification of parametric models. Two broad classes

of approaches, namely, the prediction-error minimization and correlation methods are dis-

cussed. The focus is largely on the prediction-error methods (PEM) due to their attractive

convergence properties. Certain specialized methods for specific parametric models are also

discussed. Among the correlation methods, particular attention is given to the instrumental

variable (IV) method.

21.1 RECAP

In the preceding chapter we learnt how to develop non-parametric models, i.e., models that do not

assume any specific mathematical “structure” (equation of curves) for the response of the system.

Here we study methods for estimating parametric models, which as we learnt in Chapter 4, are

a result of parametrizing the responses of the plant and noise models. The basic idea here is to

estimate the parameters instead of response coefficients. As we studied in Chapters 4 and 17, there

are significant advantages to this approach, especially that of parsimony. In frequency domain, an

added advantage is the fine resolution at which the FRF can be estimated using parametrized models

- exactly along the lines on which parametric methods score over non-parametric counterparts for

spectral density estimation. These advantages come at a price paid by the analyst in providing the

information on delay, orders, etc. which is usually obtained through non-parametric identification.

From Chapter 17 recall that primarily three broad classes of parametric model families exist,

namely, the equation-error (e.g., ARX, ARMAX), output-error and the Box-Jenkins family.

Of the three, the BJ family is the larger one containing the other two families as its members and

is described by (17.59)

y[k] = G(q

−1

,θ)u[k] + H(q

−1

,θ)e[k] =

B(q

−1

)

F(q

−1

)

u[k] +

C(q

−1

)

D(q

−1

)

e[k] (21.1)

where e[k] ∼ GWN(0, σ

2

e

) and the parameter vector θ is the set of coefficients of the finite-order

polynomials, B, C, D and F.

The prediction-error family is a generalized representation of the BJ model in which the dynamics

common to noise and plant models are exclusively highlighted

G(q

−1

,θ) =

B(q

−1

)

A(q

−1

)F (q

−1

)

; H (q

−1

,θ) =

C(q

−1

)

A(q

−1

)D(q

−1

)

(21.2)

such that F(q

−1

) and D(q

−1

) are co-prime polynomials.

Note that A(.), C(.), D(.) and F (.) are all monic polynomials, i.e., have unity as the leading

coefficient.

The one-step prediction and the prediction-error of the PE model were obtained in §18.3 and

568

Identification of Parametric Input-Output Models 569

§18.4, respectively.

ˆy[k |k − 1, θ] = H

−1

(q

−1

,θ)G(q

−1

,θ)u[k] + (1 − H

−1

(q

−1

,θ))y[k] (21.3)

=

∞

X

j=0

˜g[n]u[k − n] +

∞

X

j=1

˜

h[n]y[k − n] (21.4)

ε[k |k − 1, θ] = y[k] − ˆy[k |k − 1, θ] = H

−1

(q

−1

)(y[k] −G(q

−1

,θ)u[k]) (21.5)

where ˜g[.] and

˜

h[.] are the IR coefficients of the input and output predictor filters. In this respect, it

is useful to re-iterate a point made earlier in Chapter 18. Parametric models can also be viewed as a

parametrization of the impulse response of the predictor filters, {˜g[.]} and {

˜

h[.]}, respectively.

Note: In the presentations to follow, for convenience, the explicit dependence of the predictions, prediction errors and

transfer functions on θ is suppressed unless otherwise required.

Parametric identification problem

Given Z

N

= {y[k], u[k]}

N −1

k=0

identify the (parametrized) polynomials (A, B,C, D, F) and

variance σ

2

e

The parameter estimation problem can be suitably handled using the methods of Chapters 14

and 15. A hallmark of the parametric identification methods is that the LS, MLE and Bayesian ap-

proaches to identification can be unified under the single umbrella of what are known as prediction-

error minimization (PEM) methods. On the other hand, the MoM approach gives rise to the so-called

correlation methods. Thus, one finds two broad classes of techniques for estimation of parametric

models.

1. PEM methods: The central object of interest in this approach is the (one-step ahead) prediction-

error. A natural expectation of a good model is that it should result in a “small” prediction error.

The objective in this method is to minimize the prediction errors using the model parameters and

noise variance as vehicles. The exact form of the objective function depends on the mathematical

measure that is used to quantify the smallness of the prediction error and whether the data is

being pre-filtered.

2. Correlation methods: This is a second-order method of moments approach (recall §14.2). The

central idea is to drive the prediction errors such that they are uncorrelated with past data. As

with PEM methods, the exact set of equations depend on whether the data is pre-filtered prior

to forming the correlations. The idea can also be generalized to consider correlations between

functions of prediction errors and functions of historical data.

The rest of this chapter is devoted to a detailed study of these methods with illustrations on suitable

examples.

21.2 PREDICTION-ERROR MINIMIZATION (PEM) METHODS

The generic problem of parameter estimation in a PEM approach is formulated as follows (Ljung,

1999).

ˆ

θ

?

N

= arg min

θ

V(θ, Z

N

) (21.6a)

where V(θ, Z

N

) =

1

N

N −1

X

k=0

ˇ

l(ε(k, θ)) (21.6b)