Neural Networks with R

Avoiding overfitting in the model

The fitting of the training data causes the model to determine the weights and biases along with the activation function values. When the algorithm does too well in some training dataset, it is said to be too much aligned to that particular dataset. This leads to high variance in the output values when the test data is very different from the training data. This high estimate variance is called overfitting. The predictions are affected due to the training data provided.

There are many possible ways to handle overfitting in neural networks. The first is regularization, similar to regression. There are two kinds of regularizations:

• L1 or lasso regularization
• L2 or ridge regularization
• Max norm constraints ...