Supervised Machine Learning with Python

Tips and tricks for training a neural network

Here are some tricks that can make your life easier when you're actually training a neural network from scratch. You can stop your training a bit early to avoid overfitting. In the preceding graph, you can see there's a long tail where the error does not decrease anymore and we're still training. It's at a point around epoch 25 or 30. We could have stopped early.

Regularization and dropout are ways that can prevent your network from overfitting. Now, for extremely large data, you can do partial fits per epoch, meaning that you can fit many batches through your network for each forward pass so that you don't have to hold everything in memory. It also makes backpropagation a little easier, and different ...