Python Deep Learning Cookbook

Understanding videos with deep learning

In Chapter 7, Computer Vision, we showed how to detect and segment objects in single images. The objects in these images were fixed. However, if we add a temporal dimension to our input, objects can move within a certain scene. Understanding what is happening throughout multiple frames (a video) is a much harder task. In this recipe, we want to demonstrate how to get started when tackling videos. We will focus on combining a CNN and an RNN. The CNN is used to extract features for single frames; these features are combined and used as input for an RNN. This is also known as stacking, where we build (stack) a second model on top of another model.

For this recipe, we will be using a dataset that contains ...