All things are poisons, for there is nothing without poisonous qualities. It is only the dose which makes a thing poison.
Paracelsus, fifteenth-century Renaissance physician, botanist, alchemist, astrologer, and occultist
In this chapter, we take a look at the methods and strategies for training neural networks. We examine the following:
Obviously, this chapter can’t be comprehensive of the entire breadth of published tuning work in the space of deep learning. Our strategy was to sample the most relevant material and create a narrative that exposes you to the core concepts in deep architecture tuning. Chapter 7 then specifically focuses on tuning techniques for the most well-known architectures in deep networks:
Let’s begin with a general idea, or intuition, on how we want to approach building different neural networks with different goals.
In this chapter, we cover tuning Restricted Boltzmann Machines (RBMS) in the context of tuning DBNs.
As we get underway, you should ask yourself two questions: