Advanced Deep Learning with Python

Input and output representations

The end-to-end approach feeds raw sensor data (for example, camera images) to the ML algorithm (NN), which in turn produces the driving policy (steering angle and acceleration). In contrast, ChauffeurNet uses the middle-level input and output that we introduced in the Components of an AV system section. Let's look at the input to the ML algorithm first. This is a series of top-down (birds-eye) view 400 × 400 images, similar to the images of the CarRacing-v0 environment, but much more complex. One moment of time t is represented by multiple images, where each one contains different elements of the environment.

We can see an example of a ChauffeurNet input/output combination in the following image:

ChauffeurNet ...