Intelligent Projects Using Python

Learning the Q value function

For an RL agent to make a decision, it is important for the agent to learn the Q value function. The Q value function can be learned iteratively via Bellman's equation. When the agent starts to interact with the environment, it starts with a random state s⁽⁰⁾ and random state of Q values for every state action pair. The agent's action would also be somewhat random, since it has no state Q values to make informed decisions. For each action taken, the environment would return a reward based on which agent starts to build the Q value tables, and improves over time.

At any exposed state s^(t) at iteration t the agent would take an action a^(t) that maximizes its long-term reward. The Q table holds the long-term reward ...