Markov Processes for Stochastic Modeling, 2nd Edition

13

Controlled Markov Processes

13.1 Introduction

Controlled Markov processes are a class of processes that deal with decision making under uncertainty. These processes, which include the Markov decision process (MDP), the semi-Markov decision process (SMDP), and the partially observable Markov decision process (POMDP), can be viewed as mathematical models that are concerned with optimal strategies of a decision maker who must make a sequence of decisions over time with uncertain outcomes. These three decision processes are the subject of this chapter.

13.2 Markov Decision Processes

In MDP, a decision maker or agent can influence the state of the system by taking a sequence of actions that causes the system to optimize a predefined performance criterion. ...