Keras Reinforcement Learning Projects

Markov chain definition

As we said, a Markov chain is a mathematical model of a random phenomenon that evolves over time in such a way that the past influences the future only through the present. In other words, a stochastic model describes a sequence of possible events in which the probability of each event depends only on the state that was attained in the previous event. So, Markov chains have the property of memorylessness.

Let's consider a random process described by a sequence of random variables, X = X₀, ..., X_n, which can assume the values in a j₀, j₁,…, j_n set. We will say that it has the Markov property if the evolution of the process depends on the past only through the present—that is, the state in which we found ourselves after ...