Deep Learning with PyTorch

Applying regularization

Finding the best way to regularize the model or algorithm is one of the trickiest parts of the process, since there are a lot of parameters to be tuned. Some of the parameters that we can tune to regularize the model are:

• Adding dropout: This can be complex as this can be added between different layers, and finding the best place is usually done through experimentation. The percentage of dropout to be added is also tricky, as it is purely dependent on the problem statement we are trying to solve. It is often good practice to start with a small number such as 0.2.
• Trying different architectures: We can try different architectures, activation functions, numbers of layers, weights, or parameters inside the layers.