Deep Learning

Chapter 7. Tuning Specific Deep Network Architectures

All you really need to know for the moment is that the universe is a lot more complicated than you might think, even if you start from a position of thinking it’s pretty damn complicated in the first place.

Douglas Adams, Hitchhiker’s Guide to the Universe

In this chapter, we build on the concepts learned in Chapter 6 on general deep network tuning. We take a deeper look at tuning specific architectures such as the following:

• Convolutional Neural Networks (CNNs)
• Recurrent Neural Networks
• Deep Belief Networks (DBNs)

Given that computer vision is one of the more popular applications in the world of deep learning, let’s begin by learning to tune the CNN architecture.

Convolutional Neural Networks (CNNs)

CNNs have some general design patterns and then specific convolutional architecture design patterns. We reviewed general network design patterns in Chapter 6. In this section, we review techniques that are relevant to the CNN architecture. Most of the content for this section is focused specifically on laying out patterns of convolutional layers and pooling layers.

The convolution stage uses multiple filters to learn different features from the input to the layer, as outlined in Chapter 4. The output of this stage is transformed with an activation function—the Rectified Linear Unit (ReLU) activation function.

The Detector Stage

Some CNN literature will separate out the detector stage as its own layer or stage in the CNN architecture. ...