O'Reilly logo

Hands-On Machine Learning with Scikit-Learn and TensorFlow by Aurélien Géron

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Appendix C. SVM Dual Problem

To understand duality, you first need to understand the Lagrange multipliers method. The general idea is to transform a constrained optimization objective into an unconstrained one, by moving the constraints into the objective function. Let’s look at a simple example. Suppose you want to find the values of x and y that minimize the function f(x,y) = x2 + 2y, subject to an equality constraint: 3x + 2y + 1 = 0. Using the Lagrange multipliers method, we start by defining a new function called the Lagrangian (or Lagrange function): g(x, y, α) = f(x, y) – α(3x + 2y + 1). Each constraint (in this case just one) is subtracted from the original objective, multiplied by a new variable called a Lagrange multiplier.

Joseph-Louis Lagrange showed that if is a solution to the constrained optimization problem, then there must exist an such that is a stationary point of the Lagrangian (a stationary point is a point where all partial derivatives are equal to zero). In other words, we can compute the partial derivatives of g(x, y, α) with regards to x, y, and α; we can find the points where these derivatives are all equal to zero; and the solutions to the constrained optimization ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required