Hands-On Artificial Intelligence for IoT

Regularization

Regularization adds a term in the loss function to ensure that the cost increases as the model increases the number of features. Hence, we force the model to stay simpler. If L(X, Y) was the loss function earlier, we replace it with the following:

In the preceding, N can be L₁ norm, L₂ norm, or a combination of the two, and λ is the regularization coefficient. Regularization helps in reducing the model variance, without losing any important properties of the data distribution:

• Lasso regularization: In this case, the N is L₁ norm. It uses the modulus of weight as the penalty term N:

• Ridge regularization: In this case, the ...