In this section we discuss a special class of finite stochastic processes. We consider a sequence of finite experiments in which the outcomes and associated probabilities at each stage depend only on the outcomes and associated probabilities of the immediately preceding stage. Such a process is called a Markov process or a Markov chain. The Markov chains (processes) are named after the Russian mathematician Andrey Andreyevich Markov (1856–1922). Let’s investigate the following population movement model.
We consider a simple model of population movement between a certain city and its surrounding suburbs. For simplicity, we assume the following:
Under these two assumptions the total population in the city and the suburbs is the same every year. Assume that the demographic studies showed that during 2000, 6% of the city population move to suburbs and 2% of the suburb population move to the city. This means that 94% of the city population stays in the city and 98% of the suburb population stays in the suburb.