Chapter 5

Time Series Analysis

Time series analysis has become an indispensable theoretical tool in

financial and economic research. Section 5.1 is devoted to the com-

monly used univariate autoregressive and moving average models.

The means for modeling trends and seasonality effects are described

in Section 5.2. The processes with non-stationary variance (condi-

tional heteroskedasticity) are discussed in Section 5.3. Finally,

the specifics of the multivariate time series are introduced in

Section 5.4.

5.1 AUTOREGRESSIVE AND MOVING AVERAGE

MODELS

5.1.1 A

UTOREGRESSIVE MODEL

First, we shall consider a univariate time series y(t) for a process

that is observed at moments t ¼ 0, 1, ..., n (see, e.g., [1, 2]). The time

series in which the observation at moment t depends linearly on

several lagged observations at moments t 1, t 2, ...,t p

y(t) ¼ a

1

y(t 1) þ a

2

y(t 2) þ ...þ a

p

y(t p) þ e(t), t > p(5:1:1)

is called the autoregressive process of order p, or AR(p). The term e(t) in

(5.1.1) is the white noise that satisfies the conditions (4.2.6). The lag

43

operator L

p

¼ y(t p) is often used for describing time series. Note that

L

0

¼ y(t). Equation (5.1.1) in terms of the lag operator has the form

A

p

(L)y(t) ¼ e(t) (5:1:2)

where

A

p

(L) ¼ 1 a

1

L a

2

L

2

... a

p

L

p

(5:1:3)

The operator A

p

(L) is called the AR polynomial in lag operator of

order p. Let us consider AR(1) that starts with a random shock. Its

definition implies that

y(0) ¼ e(0), y(1) ¼ a

1

y(0) þ e(1),

y(2) ¼ a

1

y(1) þ e(2) ¼ a

1

2

e(0) þ a

1

e(1) þ e(2), ...

Hence, by induction,

y(t) ¼

X

t

i¼0

a

1

i

e(t i) (5:1:4)

Mean and variance of AR(1) equal, respectively

E[y(t)] ¼ 0, Var[y(t)] ¼ s

2

=(1 a

1

2

), (5:1:5)

Obviously, the contributions of the ‘‘old’’ noise converge with time to

zero when ja

1

j < 1. As a result, this process does not drift too far from

its mean. This feature is named mean reversion.

The process with a

1

¼ 1 is called the random walk

y(t) ¼ y(t 1) þ e(t) (5:1:6)

In this case, equation (5.1.4) reduces to

y(t) ¼

X

t

i¼0

e(t i)

The noise contributions to the random walk do not weaken with time.

Therefore, the random walk does not exhibit mean reversion. Now,

consider the process that represents the first difference

x(t) ¼ y(t) y(t 1) ¼ e(t) (5:1:7)

Obviously, past noise has only transitory character for the process

x(t). Therefore, x(t) is mean-reverting. Some processes must be

44 Time Series Analysis

differenced several times in order to exclude non-transitory noise

shocks. The processes differenced d times are named integrated of

order d and denoted as I(d). The differencing operator is used for

describing an I(d) process

D

i

d

¼ (1 L

i

)

d

,j,d¼ ..., 2, 1, 0, 1, 2 ... (5:1:8)

If an I(d) process can be reduced to AR(p) process while applying the

differencing operator, it is named ARI(p, d) process and has the form:

D

1

d

y(t) a

1

D

1

d

y(t 1) ... a

p

D

1

d

y(t p) ¼ e(t), t p þ d

(5:1:9)

Note that differencing a time series d times reduces the number of

independent variables by d, so that the total number of independent

variables in ARI(p, d) within the sample with n observations equals

n p d.

The unit root is another notion widely used for discerning perman-

ent and transitory effects of random shocks. It is based on the roots of

the characteristic polynomial for the AR(p) model. For example,

AR(1) has the characteristic polynomial

1 a

1

z ¼ 0(5:1:10)

If a

1

¼ 1, then z ¼ 1 and the characteristic polynomial has the

unit root. In general, the characteristic polynomial roots can have

complex values. The solution to equation (5.1.10) is outside the unit

circle (i.e., z > 1) when a

1

< 1. It can be shown that all solutions for

AR(p) are outside the unit circle when

1 a

1

z a

2

z

2

... a

p

z

p

¼ 0(5:1:11)

5.1.2 MOVING AVERAGE MODELS

A model more general than AR(p) contains both lagged observa-

tions and lagged noise

y(t) ¼ a

1

y(t 1) þ a

2

y(t 2) þ ...þ a

p

y(t p) þ e(t)

þ b

1

e(t 1) þ b

2

e(t 2) þ ...þ b

q

e(t q) (5:1:12)

This model is called autoregressive moving average model of order

(p,q), or simply ARMA(p,q). Sometimes modeling of empirical data

Time Series Analysis 45

Get *Quantitative Finance for Physicists* now with the O’Reilly learning platform.

O’Reilly members experience live online training, plus books, videos, and digital content from nearly 200 publishers.