Machine Learning for Developers

L1 versus L2 properties

Now it's time to do a head-to-head comparison between the two types of loss function:

• Robustness: L1 is a more robust loss function, which can be expressed as the resistance of the function when being affected by outliers, which projects a quadratic function to very high values. Thus, in order to choose an L2 function, we should have very stringent data cleaning for it to be efficient.
• Stability: The stability property assesses how much the error curve jumps for a large error value. L1 is more unstable, especially for non-normalized datasets (because numbers in the [-1, 1] range diminish when squared).
• Solution uniqueness: As can be inferred by its quadratic nature, the L2 function ensures that we will have a unique ...