Practical Deep Learning

10EXPERIMENTS WITH NEURAL NETWORKS

In Chapter 9, we discussed the theory behind neural networks. In this chapter, we’ll trade equations for code and run a number of experiments designed to increase our intuition regarding the essential parameters of neural networks: architecture and activation functions, batch size, base learning rate, training set size, L2 regularization, momentum, weight initialization, feature ordering, and the precision of the weights and biases.

To save space and eliminate tedious repetition, we won’t show the specific code for each experiment. In most cases, the code is only trivially different from the previous example; ...