Convexity & optimization landscapes

This surface has two dips separated by a ridge. Drag along the ridge itself, right through the middle — the gradient can shrink to nearly nothing there too, even though it obviously isn't the bottom of anything.

A function is convex if the Hessian's eigenvalues are non-negative everywhere — informally, if it curves upward (or stays flat) in every direction, at every point. When that fails, a critical point (where $\nabla f = 0$) needs a second test to classify:

• $H_f$ — the Hessian of $f$ at the critical point being classified.
• $f_{xx}, f_{yy}$ — the two pure second partials (curvature along each axis).
• $f_{xy}$ — the mixed second partial (how curvature along one axis changes as the other moves).

1. det > 0 and fxx > 0: a minimum

   Both axes curve upward and agree with each other — a genuine local bottom.

2. det > 0 and fxx < 0: a maximum

   Both axes curve downward together — a genuine local top.

3. det < 0: a saddle point

   The axes disagree — curving up along one direction and down along another. The gradient still vanishes here, but it's neither a min nor a max: this is exactly what happened on the ridge above.

4. Convex functions never produce a saddle

   If $f$ is convex everywhere, $\det H_f \geq 0$ and $f_{xx}\geq 0$ hold at every point, so this test can only ever report "minimum" — which is precisely why convexity was such a strong guarantee in the first place.

On the convex bowl, every critical point the test finds is a minimum — there's no other option. Switch to the non-convex landscape and settle near the ridge at the center: same test, same formula, and this time it reports "saddle."

Let $f(x,y) = (x^2-2)^2+y^2$, which has three critical points: the ridge at the origin, and two wells at $x=\pm\sqrt2$.

1. Confirm the origin is critical

   Differentiate via the chain rule — the outer square, times the inner derivative:

   • $\partial f/\partial x = 2(x^2-2)\cdot\frac{\partial}{\partial x}(x^2-2) = 2(x^2-2)\cdot 2x = 4x(x^2-2)$
   • $\partial f/\partial y = 2y$

   $$\nabla f(0,0) = \big(4(0)(0^2-2),\ 2(0)\big) = (0,0)$$
2. Classify it with the Hessian test

   Differentiate the two first partials again:

   • From $\partial f/\partial x = 4x^3-8x$: $f_{xx}=\partial/\partial x[4x^3-8x]=12x^2-8$
   • From that same expression: $f_{xy}=\partial/\partial y[4x^3-8x]=0$ (no $y$ appears in it)
   • From $\partial f/\partial y = 2y$: $f_{yy}=\partial/\partial y[2y]=2$

   At $x=0$: $f_{xx}=12(0)^2-8=-8$, so $\det H_f = (-8)(2) - 0^2 = -16 < 0$ — a saddle, exactly as the drag confirmed.

3. Classify a well the same way

   At $x=\sqrt2$: $f_{xx}=12(2)-8=16>0$, so $\det H_f = (16)(2)=32>0$ and $f_{xx}>0$ — a minimum, with $f(\sqrt2,0)=0$, the global minimum value.