Hands-On Reinforcement Learning for Games

Exploring SARSA

In this chapter, we continue with our focus on Temporal Difference Learning (TDL) and expand on it from TD (0) to multi-step TD and beyond. We will look at a new method of Reinforcement Learning (RL) called SARSA, explore what it is, and how it differs from Q-learning. From there, we will look at a few examples with new continual control learning environments from Gym. Then, we will move to a deeper understanding of TDL and introduce concepts called TD lambda (λ) and eligibility traces. Finally, we will finish off this chapter by looking at an example of SARSA.

For this chapter, we will extend our discussion of TDL and uncover State Action Reward State Action (SARSA), continuous action spaces, TD (λ), eligibility traces, and ...