Programming Machine Learning

Chapter 11Training the Network

In the previous chapter, we wrote a functioning neural network—or at least half of it. The network’s prediction code is done: it passes data through the model, and churns out labels. However, that process also requires a set of weights, and we still didn’t write the code that finds those weights. We’ll do it in this chapter by implementing the train function of our neural network.

In the early years of neural networks, training was a tough nut to crack. AI experts even questioned whether they could be trained at all. The answer came in the early 1970s, when researchers found a way to calculate the gradient of a network with an algorithm called backpropagation—or “backprop” for friends.

Chances are, you’ll never ...