O'Reilly logo

Machine Learning by Sergios Theodoridis

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Appendix C

Hints on Constrained Optimization

Chapter Outline

C.1 Equality Constraints 1023

C.2 Inequality Constraints 1025

The Karush-Kuhn-Tucker (KKT) conditions 1025

Min-Max duality 1026

Saddle point condition 1027

Lagrangian duality 1027

Convex programming 1028

Wolfe dual representation 1029

References 1029

C.1 Equality Constraints

We will first focus on linear equality constraints and then generalize to the nonlinear case. The problem is cast as

minθJ(θ),s.t.Aθ=b,

si1_e

where A is an m × l matrix and b, θ are m × 1 and l × 1 vectors, respectively. It is assumed that the cost function J(θ) is twice continuously differentiable and it is, in general, ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required