Programming Machine Learning

Learning, Visualized

To accelerate training, we need to understand in more detail how it works. Let’s take a deeper look at how the neural network’s loss and accuracy change during training.

We know that during training, the loss goes down, and the accuracy goes up. Staring at those numbers, however, doesn’t tell the whole story. To visualize the loss and accuracy, I hacked the neural network to return two lists—the histories of the loss and the accuracy, stored at each iteration. I trained the network to collect the two histories, and then plotted them over 30 iterations, as illustrated in the following graphs (if you want to run the code yourself, you can find it in the code download in batching/plot_loss.py):

These diagrams show what we expect: ...