Hands-On Reinforcement Learning for Games

Encoding visual state

Fortunately for DRL, the concept of learning from an image has been the center of ongoing research into DL for over 30 years. DL has taken this concept from being able to recognize handwritten digits to being able to detect object position and rotation to understanding the human pose. All of this is done by feeding raw pixels into a deep learning network and it being taught (or teaching itself) how to encode those images to some answer. We will use these same tools in this chapter, but before we do, let's understand the fundamentals of taking an image and feeding it into a network. An example of how you may do this is shown in the following diagram:

Dissecting an image for input into DL

In the preceding diagram, the ...