Theory of Probability

9An Introduction to Other Types of Stochastic Process

9.1 Markov Processes

9.1.1. The cases treated so far have been considered at some length, this being a convenient way in which to introduce various of the basic notions and most frequently used techniques. They are, however, nothing more than examples of the simplest and most special form of random process; that is, the linear form, or, more explicitly, the homogeneous process with independent increments. We now give some of the basic properties of other cases of interest, although, given the limits of the present work, the treatment will necessarily be brief.

Processes for which ‘given the present, the future is independent of the past’, or, alternatively, ‘the future depends on the past only through the present’ are called Markov processes. Processes with independent increments are a special case (they are even independent of the present, and the process depends on the latter only through the fact that the future increment, Y(t) − Y(t₀), is added on to the present value Y(t₀)); the Markov property is much less restrictive.

The name derives from the fact that Markov considered this property in a particular discrete situation (involving probabilities of ‘linked’ events, whence Markov chains). To give a simple example, let us consider a function Y_n, taking only a finite number of values, 1, 2,…, r, say. For a physical interpretation, which may be more expressive, we could think of it as a ‘system’ which can be in any one ...