Skip to Content
Markov Decision Processes in Artificial Intelligence
book

Markov Decision Processes in Artificial Intelligence

by Olivier Sigaud, Olivier Buffet
March 2010
Intermediate to advanced
480 pages
12h 28m
English
Wiley
Content preview from Markov Decision Processes in Artificial Intelligence

Chapter 15

Operations Planning 1

15.1. Operations planning

This chapter deals with the application of MDPs to operations1 planning problems. Such problems arise in areas ranging from space exploration (rovers, satellites, telescopes) to project management and military operations. Automated planning [GHA 04, REG 04] is a branch of Artificial Intelligence. It aims at building general-purpose systems capable of choosing and organizing the operations to perform, in such a way as to reach given objectives at least cost. Here, we examine complex planning problems. These not only require concurrent operation execution and the explicit modeling of time and operation durations. They also make it necessary to account for uncertainty about the effects of operations and about the time at which they occur. Such problems are known as probabilistic temporal planning problems [ABE 06, ABE 07b, LIT 05, MAU 05, MAU 07]. We define them in an intuitive way before providing a formal definition. We then focus on their modeling as an MDP and on their resolution using variants of algorithms presented in the previous chapters.

15.1.1. Intuition

We start with an intuitive presentation of the problem illustrated by a space exploration example. In our scenario, a Mars rover must carry out several experiments on various sites and acquire scientific data which it must transmit to Earth [BRE 02] (see also Chapter 14).

15.1.1.1. Problem features

An operations planning problem is characterized by:

– an environment ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

AirBnbBlueOriginElectronic ArtsHomeDepotNasdaqRakutenTata Consultancy Services

QuotationMarkO’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.
Julian F.
Head of Cybersecurity
QuotationMarkI wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.
Addison B.
Field Engineer
QuotationMarkI’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.
Amir M.
Data Platform Tech Lead
QuotationMarkI'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
Mark W.
Embedded Software Engineer

You might also like

Algorithmic Management: The Role of AI in Managing Workforces

Algorithmic Management: The Role of AI in Managing Workforces

Mohammad Hossein Jarrahi, Mareike Möhlmann, Min Kyung Lee

Publisher Resources

ISBN: 9781118620106Purchase book