Python Deep Learning Cookbook

Tuning the loss function

While training a neural network for a supervised learning problem, the objective of the network is to minimize the loss function. The loss function — also known as error, cost function, or opimization function–compares the prediction with the ground truth during the forward pass. The output of this loss function is used to optimize the weights during the backward pass. Therefore, the loss function is crucial in training the network. By setting the correct loss function, we force the network to optimize towards the desired predictions. For example, for imbalanced datasets we need a different loss function.

In the previous recipes, we've used mean squared error (MSE) and categorical cross entropy as loss functions. ...