At this point, we want to test the TD(λ) algorithm with a slightly more complex tunnel environment. In fact, together with the absorbing states, we will also consider some intermediate positive states, which can be imagined as checkpoints. An agent should learn the optimal path from any cell to the final state, trying to pass through the highest number of checkpoints possible. Let's start by defining the new structure:
import numpy as npwidth = 15height = 5y_final = width - 1x_final = height - 1y_wells = [0, 1, 3, 5, 5, 6, 7, 9, 10, 11, 12, 14]x_wells = [3, 1, 2, 0, 4, 3, 1, 3, 1, 2, 4, 1]y_prizes = [0, 3, 4, 6, 7, 8, 9, 12]x_prizes = [2, 4, 3, 2, 1, 4, 0, 2]standard_reward = -0.1tunnel_rewards ...
