Deep Learning

Chapter 6. Tuning Deep Networks

All things are poisons, for there is nothing without poisonous qualities. It is only the dose which makes a thing poison.

Paracelsus, fifteenth-century Renaissance physician, botanist, alchemist, astrologer, and occultist

Basic Concepts in Tuning Deep Networks

In this chapter, we take a look at the methods and strategies for training neural networks. We examine the following:

• Matching network architecture to the problem at hand
• The basics of hyperparameter tuning
• Better understanding the process of learning

Obviously, this chapter can’t be comprehensive of the entire breadth of published tuning work in the space of deep learning. Our strategy was to sample the most relevant material and create a narrative that exposes you to the core concepts in deep architecture tuning. Chapter 7 then specifically focuses on tuning techniques for the most well-known architectures in deep networks:

• Deep Belief Networks (DBNs)
• Convolutional Neural Networks (CNNs)
• Recurrent Neural Networks

Let’s begin with a general idea, or intuition, on how we want to approach building different neural networks with different goals.

Tuning Restricted Boltzmann Machines

In this chapter, we cover tuning Restricted Boltzmann Machines (RBMS) in the context of tuning DBNs.

An Intuition for Building Deep Networks

As we get underway, you should ask yourself two questions:

• What kind of input data will I be modeling?
• What is the output I want to know after this model is constructed? ...