Simulation with Python: Develop Simulation and Modeling in Natural Sciences, Engineering, and Social Sciences

© The Author(s), under exclusive license to APress Media, LLC, part of Springer Nature 2022

R. Li, A. NakanoSimulation with Pythonhttps://doi.org/10.1007/978-1-4842-8185-7_2

2. Markov Chain, a Peek into the Future

Rongpeng Li¹   and Aiichiro Nakano¹

(1)

Los Angeles, CA, USA

 

In this chapter, we continue our exploration in the world of simulation. Different from the previous Monte Carlo simulation where the scenario is purely static, which means there is no dynamics in the simulation, we are going to study dynamics of a system.

The Markov chain, specifically the discrete-time Markov chain, is named after Russian mathematician Andrey Andreyevich Markov. He is a pioneer in the study of stochastic processes and the first to introduce the concept of Markov ...