Chapter 12Markov Chains
In this chapter, we start the study of some the most popular class of models suitable for real-life situations.
12.1 Basic Concepts for Markov Chains
12.1.1 Definition
Consider a set of outcomes 
which is finite or countable. 
is called the states space. It is convenient to represent the set 
as the nonnegative integers 
(any discrete set may be put into a bijection with this set). Consider a process 
whose components 
take values in this set 
. We will say that the process X is in state 
at time n if 
.
We next consider a matrix
with the elements
Such a matrix is often ...
Get Probability and Stochastic Processes now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
