Keras Reinforcement Learning Projects

Transition diagram

A very intuitive alternative to the description of a Markov chain through a transition matrix is associating an oriented graph (transition diagram) to a Markov chain to which the following two statements apply:

• Vertices are labeled by the S₁, S₂,…, S_n states (or, briefly, from the indices 1, 2, …, n of the states)
• There is a directed edge that connects the S_i vertex to the S_j vertex if and only if the probability of transition from S_i to S_j is positive (a probability that is in turn used as a label of the edge itself)

It is clear that the transition matrix and transition diagram provide the same information regarding the same Markov chain. To understand this duality, we can look at a simple example. Say that we have a ...