O'Reilly logo

Deep Learning by Adam Gibson, Josh Patterson

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Chapter 6. Tuning Deep Networks

All things are poisons, for there is nothing without poisonous qualities. It is only the dose which makes a thing poison.

Paracelsus, fifteenth-century Renaissance physician, botanist, alchemist, astrologer, and occultist

Basic Concepts in Tuning Deep Networks

In this chapter, we take a look at the methods and strategies for training neural networks. We examine the following:

  • Matching network architecture to the problem at hand
  • The basics of hyperparameter tuning
  • Better understanding the process of learning

Obviously, this chapter can’t be comprehensive of the entire breadth of published tuning work in the space of deep learning. Our strategy was to sample the most relevant material and create a narrative that exposes you to the core concepts in deep architecture tuning. Chapter 7 then specifically focuses on tuning techniques for the most well-known architectures in deep networks:

  • Deep Belief Networks (DBNs)
  • Convolutional Neural Networks (CNNs)
  • Recurrent Neural Networks

Let’s begin with a general idea, or intuition, on how we want to approach building different neural networks with different goals.

Tuning Restricted Boltzmann Machines

In this chapter, we cover tuning Restricted Boltzmann Machines (RBMS) in the context of tuning DBNs.

An Intuition for Building Deep Networks

As we get underway, you should ask yourself two questions:

  • What kind of input data will I be modeling?
  • What is the output I want to know after this model is constructed? ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required