Hands-On Artificial Intelligence for Beginners

Using cross-entropy for binary classification problems

While we can use MSE for regression problems, we need to use a different type of loss for classification problems. For that, we use a function known as cross-entropy. Cross entropy measure the performance of a classification model whose outputs are between 0 and 1. A low cross entropy means that a predicted classification is similar to the actual classification. A high cross entropy, on the other hand, means that a predicted classification is different from the real classification. 

 It's also known as Bernoulli negative log-likelihood and Binary cross-entropy:

def CrossEntropy(yHat, y):    if yHat == 1:         return -log(y)    else:        return -log(1 - y)

Defining loss functions for classification problems ...