Programming Machine Learning

Cross Entropy

So far, we used the log loss formula for our binary classifiers. We even used the log loss when we bundled ten binary classifiers in a multiclass classifier (in Chapter 7, ​The Final Challenge​). In that case, we added together the losses of the ten classifiers to get a total loss.

While the log loss served us well so far, it’s time to switch to a simpler formula—one that’s specific to multiclass classifiers. It’s called the cross-entropy loss, and it looks like this:

Here’s the cross-entropy loss in code form:

def loss(Y, y_hat):
    return -np.sum(Y * np.log(y_hat)) / Y.shape[0]

Check out “Grokking the Cross-Entropy Loss” on ...