Principles of System Identification

11

Spectral Representations of Random

Processes

In this chapter, we study the spectral representations of random processes. Specifically, the

focus is on the definitions of power spectral density for random signals, the Wiener-Khinchin

theorem and the important spectral factorization result. Bivariate frequency-domain mea-

sures, namely, cross-power spectrum, coherence and partial coherence are also discussed. A

study of this chapter reinforces concepts in previous chapters and offers a strong foundation

for estimation of frequency-domain models, time-delay estimation and input design.

11.1 INTRODUCTION

Spectral distributions and densities for random signals cannot be constructed using the same line

of approach as that for deterministic signals owing to an important fact: the Fourier transform of

a random signal does not exist. The reason is that a random signal neither satisfies the finite 1-

norm requirement nor the finite 2-norm requirement. Nevertheless, the theoretical developments for

deterministic signals in the previous section, specifically the concept of Fourier transforms and the

Wiener-Khinchin theorem are foundational in developing a similar theory for random processes.

Random signals (with bounded amplitude) are in general, aperiodic power signals, for which

no Fourier transform or series definitions exist. At a later stage, we shall study a special class of

random processes known as harmonic processes, which are mean-square periodic.

The aperiodic nature of random signals encourages us to postulate the existence of a spectral

density. In the following sections, we shall learn how spectral densities are defined for stationary

random processes, the conditions under which they exist and their theoretical aspects. These sec-

tions summarize an encouraging and important fact - spectral densities also exist for (a large class

of) stationary random processes and they are related to the auto-covariance functions through the

Fourier transforms, thereby exactly resembling the case for deterministic signals.

Historical note

The theory pertaining to spectral representations of random processes is largely due to a culmination

of pioneering works in the fields of mathematics and signal processing during the years 1920-1950

(due to Wiener, Khinchin, Wold, Herglotz, Bochner, Kolmogorov, Cramer and Wintner).

Wiener’s generalized harmonic analysis (GHA) is one of the foundational results in this context

(Benedetto, 1997; Wiener, 1930). Another fundamental result known as the Wiener-Khinchin the-

orem1 (also available as Herglotz or Bochner theorem in functional analysis) for continuous-time

stochastic processes along with the associated Wold’s theorem for discrete-time processes constitute

the milestone results in spectral representations of stochastic processes.

1Certain historical critiques term this as a misnomer with a note that Khinchin established the result first, which Wiener

used later in 1950.

270