Hands-On Q-Learning with Python

Updating the Q-values

The update function modifies the Q-values according to our familiar Bellman equation:

• NewQ(s,a): New Q value for that state and that action
• Q(s,a): Current Q value
• α: Learning Rate
• R(s,a): Reward for taking that action at that state
• γ: Discount Rate
• maxQ^'(s^',a^'): maximum expected future reward given the new s' and all possible actions at that new state

We use the alpha and gamma values we declared earlier, set the new Q-value for the current state based on the maximum Q-value for the next state, and fit the model to our new state and Q-value:

   def update(self, state, action, reward, next_state, done): q_update = reward ...