Hands-On Mathematics for Deep Learning

Loss functions

We know that we are trying to approximate a function and we are trying to get as close as possible to the true function. To do this, we need to define a loss function—we have many to choose from. The following are the main ones that are used in practice:

• , known as mean squared error
• , known as mean absolute error
• , known as square loss
• , known as hinge loss
• , known as cross-entropy loss
• , known as Huber loss

We will revisit them ...