O'Reilly logo

Hands-On Natural Language Processing with Python by Rajalingappaa Shanmugamani, Rajesh Arumugam

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

L1 and L2 normalization

L1 and L2 normalization are common regularization techniques that control how much the weights can grow or shrink in the network during training. It has the effect of not giving too much importance to a specific feature, similar to dropout. In L1 regularization, the loss function increases in direct proportion to the size of the weights, whereas in L2 normalization, it increases in proportion to the square of the weights.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required