Hands-On Deep Learning with Go

Loss functions

Loss functions play an important part in training our network. We haven't discussed them in much detail, but their role is to tell our model when it gets things wrong, so it can learn from its mistakes.

In this example, we are using a version of cross-entropy loss that has been modified to be as efficient as possible.

It should be noted that cross-entropy loss would typically be expressed in pseudocode, such as the following:

crossEntropyLoss = -1 * sum(actual_y * log(predicted_y))

However, in our case, we are going for a simpler version:

loss = -1 * mean(actual_y * predicted_y)

So, we are implementing the loss function as follows:

losses, err := gorgonia.HadamardProd(m.out, y)if err != nil {    log.Fatal(err)}cost := gorgonia.Must(gorgonia.Mean(losses)) ...