Hands-On Mathematics for Deep Learning

Lagrange multipliers

When solving constrained optimization problems, it is best to include the constraints in the objective function. This way, anything that is not included in the constraints is not considered a minimum.

Let's revisit our earlier problem:

We'll call our constraint C.

So, we define the Lagrangian of C as follows:

Here, and is known as the Lagrange multiplier.

When our constraint is satisfied, then and . By minimizing L over ...