Hands-On Simulation Modeling with Python

Chapter 5: Simulation-Based Markov Decision Processes

Markov Decision Processes (MDPs) model decision-making in situations where outcomes are partly random and partly under the control of a decision maker. An MDP is a stochastic process characterized by five elements: decision epochs, states, actions, transition probability, and reward. The characteristic elements of a Markovian process are the states in which the system finds itself and the available actions that the decision maker can carry out on those states. These elements identify two sets: the set of states in which the system can be found and the set of actions available for each specific state. The action chosen by the decision maker determines a random response from the system, which ...