Hands-On Q-Learning with Python

Policy agents versus value agents

Let's discuss one primary difference between the model-free agent that we built in Chapter 4, Teaching a Smartcab to Drive Using Q-Learning, and the Q-networks we'll be building here and in the next chapter. We've brought up this topic briefly before, but it's important to understand it at a higher level:

• Model-free Q-learning agents are value agents; they do not attempt to model the agent's policy or state-action function. They only represent the actual Q-values in their implementation, such as in a Q-table.
• Policy agents, such as the ones we'll be implementing with deep Q-networks, model the agent's policy:

The primary practical difference is that policy agents, unlike value agents, develop the ability ...