The Art of Reinforcement Learning: Fundamentals, Mathematics, and Implementations with Python

© The Author(s), under exclusive license to APress Media, LLC, part of Springer Nature 2023

M. HuThe Art of Reinforcement Learninghttps://doi.org/10.1007/978-1-4842-9606-6_5

5. Temporal Difference Learning

Michael Hu¹  

(1)

Shanghai, Shanghai, China

 

In the previous chapter (Chap. 4), we introduced Monte Carlo (MC) methods for reinforcement learning, which allow an agent to learn from its own experience without relying on a model of the environment. However, MC methods are limited to episodic reinforcement learning problems, where the agent interacts with the environment in discrete episodes. In this chapter, we introduce temporal difference (TD) learning, a class of algorithms that generalize MC methods to support both episodic and continuing reinforcement ...