O'Reilly logo

Probability and Stochastic Processes by Ionut Florescu

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Chapter 12Markov Chains

In this chapter, we start the study of some the most popular class of models suitable for real-life situations.

12.1 Basic Concepts for Markov Chains

12.1.1 Definition

Consider a set of outcomes c12-math-0001 which is finite or countable. c12-math-0002 is called the states space. It is convenient to represent the set c12-math-0003 as the nonnegative integers c12-math-0004 (any discrete set may be put into a bijection with this set). Consider a process c12-math-0005 whose components c12-math-0006 take values in this set c12-math-0007. We will say that the process X is in state c12-math-0009 at time n if c12-math-0011.

We next consider a matrix

with the elements

Such a matrix is often ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required