Mastering Machine Learning Algorithms

TD(λ) algorithm

In the previous chapter, we introduced the temporal difference strategy, and we discussed a simple example called TD(0). In the case of TD(0), the discounted reward is approximated by using a one-step backup. Hence, if the agent performs an action a_t in the state s_t, and the transition to the state s_t+1 is observed, the approximation becomes the following:

If the task is episodic (as in many real-life scenarios) and has T(e_i) steps, the complete backup for the episode e_i is as follows:

The previous expression ends when the MDP ...