TensorFlow 2.0 Quick Start Guide

Loss functions

A loss function (that is, an error measurement) is a necessary part of the training of an ANN. It is a measure of the extent to which the calculated output of a network during training differs from its required output. By differentiating the loss function, we can find a quantity with which to adjust the weights of the connections between the layers so as to make the calculated output of the ANN more closely match the required output.

The simplest loss function is the mean squared error:

,

Here, y is the actual label value, and is the predicted label value.

Of particular note is the categorical cross-entropy loss function, which ...