Constrained optimization & Lagrange multipliers

The dashed line is a hard constraint: every point on it satisfies $x+y=4$. The circles are level sets of $f(x,y)=x^2+y^2$ — the smaller the circle, the smaller $f$ is there. Slide along the line and watch the circle passing through the point grow and shrink.

At the constrained optimum, the objective's level curve is exactly tangent to the constraint — their gradients point the same direction:

• $f(x,y)$ — the objective being minimized (or maximized).
• $g(x,y)=0$ — the equality constraint the solution must satisfy.
• $\lambda$ — the Lagrange multiplier: the scalar that makes the two gradients equal.

1. Why tangency is the condition

   If the level curve crossed the constraint instead of touching it, you could slide a little further along the constraint and keep decreasing $f$ — so a crossing point can never be optimal. Only where the curve is tangent to the line is there nowhere better left to slide.

2. Tangent curves share a normal direction

   Two curves are tangent exactly when their gradients (normal directions) are parallel — which is what $\nabla f = \lambda\nabla g$ says algebraically.

3. λ measures sensitivity to the constraint

   $\lambda$ isn't just bookkeeping: it says how much the optimal $f$ would change per unit of slack if the constraint (here, the "4") were relaxed.

The accent arrow is $\nabla f$ at the current point; the dashed-line-colored arrow is the fixed direction of $\nabla g=(1,1)$. Slide until they point the same way — that's the only spot on the whole line where they agree, and it's exactly where the level circle stops crossing the line and just touches it.

Minimize $f(x,y)=x^2+y^2$ subject to $g(x,y)=x+y-4=0$:

1. Write the Lagrange condition componentwise

   $\nabla f = (2x, 2y)$ and $\nabla g=(1,1)$, so $\nabla f=\lambda\nabla g$ means $2x=\lambda$ and $2y=\lambda$ — which forces $x=y$.

2. Combine with the constraint

   Substituting $x=y$ into $x+y=4$ gives $2x=4$, so $x=y=2$ and $\lambda=2x=4$.

3. Check a non-optimal point for contrast

   At $(0,4)$: $\nabla f=(0,8)$. Matching components gives $\lambda=0$ from $x$ but $\lambda=8$ from $y$ — they disagree, so $(0,4)$ is not a critical point of the constrained problem, and indeed $f(0,4)=16 > f(2,2)=8$.