Deep Learning with PyTorch

Large model enough to overfit

Once you have a model that has enough capacity to beat your baseline score, increase your baseline capacity. A few simple tricks to increase the capacity of your architecture are as follows:

• Add more layers to your existing architecture
• Add more weights to the existing layers
• Train it for more epochs

We generally train the model for an adequate number of epochs. Stop it when the training accuracy keeps increasing and the validation accuracy stops increasing and probably starts dropping; that's where the model starts overfitting. Once we reach this stage, we need to apply regularization techniques.

Remember, the number of layers, size of layers, and number of epochs may change from problem to problem. A smaller ...