Principles of System Identification

19

Identification of Parametric Time-Series

Models

Estimators of time-series models are discussed. The primary objectives are to provide the

working principles of these methods and their implementation. Concepts learnt from this

chapter are useful for disturbance (noise) modeling and estimation of spectral densities.

19.1 INTRODUCTION

In Chapter 9 we studied different time-series models for linear stationary random processes. A

general description is given by the ARMA model (or ARIMA for integrating processes). In this

chapter, we shall learn how to estimate these models using the methods of Chapters 14 and 15.

Historically, time-series modeling precluded identification by a few decades. Furthermore recall

from our discussions on previous chapters that estimating models for identification is essentially

identical to developing a time-series model with exogenous effects. Therefore, it is natural that we

familiarize ourselves with the methods and procedures for estimation of time-series models first.

The resulting model is useful in (i) explaining the effects of disturbances (noise modeling) and (ii)

estimation of power spectral densities (recall §16.5.7).

A variety of methods are available for estimating ARMA models, almost all of which emerge

from systematically applying the techniques and estimators in Chapters 13 through 15. However,

the differences in the properties of the AR, MA and the ARMA model structures attract different

estimation methods. For instance, auto-regressive models result in linear predictors; therefore, a

linear OLS method delivers the goods. On the other end, MA models are described by non-linear

predictors calling for a non-linear LS method, which is computationally more intense and results in

local optima. The linear nature of the AR predictors also attracts a few other specialized methods.

The historical nature and the applicability of this topic is such that numerous texts and survey/tu-

torial articles dedicated to this topic have been written (Box, Jenkins and Reinsel, 2008; Brockwell,

2002; Chatfield, 2004; Priestley, 1981; Shumway and Stoffer, 2006). Therefore, it is neither the ob-

jective of this short chapter nor is it feasible to give an in-depth treatment of the subject. The primary

aims of this chapter are to (i) highlight the working principles of popular methods, (ii) discuss the

properties of the resulting estimators and (iii) demonstrate their implementation (in MATLAB). The

chapter begins with the estimation of AR models in §19.2 and gradually develops into the estimation

of ARMA models in §19.4. In the concluding part, we discuss the estimation of ARIMA models.

19.2 ESTIMATION OF AR MODELS

The AR estimation problem is stated as follows. Given N observations of a stationary process {v[k]},

k = 0,··· , N − 1, fit an AR(P) model (recall (9.28)).

v[k] =

P

X

j=1

(−d

j

)v[k − j] + e[k] (9.28 revisited)

One of the first methods that were developed to estimate AR models was the Yule-Walker method

based on the Yule-Walker equations (recall §9.5.2). The Y-W method belongs to the class of method

520