13

Controlled Markov Processes

13.1 Introduction

Controlled Markov processes are a class of processes that deal with decision making under uncertainty. These processes, which include the Markov decision process (MDP), the semi-Markov decision process (SMDP), and the partially observable Markov decision process (POMDP), can be viewed as mathematical models that are concerned with optimal strategies of a decision maker who must make a sequence of decisions over time with uncertain outcomes. These three decision processes are the subject of this chapter.

13.2 Markov Decision Processes

In MDP, a decision maker or agent can influence the state of the system by taking a sequence of actions that causes the system to optimize a predefined performance criterion. ...

Get Markov Processes for Stochastic Modeling, 2nd Edition now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.