O'Reilly logo

Markov Processes for Stochastic Modeling, 2nd Edition by Oliver Ibe

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

13

Controlled Markov Processes

13.1 Introduction

Controlled Markov processes are a class of processes that deal with decision making under uncertainty. These processes, which include the Markov decision process (MDP), the semi-Markov decision process (SMDP), and the partially observable Markov decision process (POMDP), can be viewed as mathematical models that are concerned with optimal strategies of a decision maker who must make a sequence of decisions over time with uncertain outcomes. These three decision processes are the subject of this chapter.

13.2 Markov Decision Processes

In MDP, a decision maker or agent can influence the state of the system by taking a sequence of actions that causes the system to optimize a predefined performance criterion. ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required