Part I — Calculus, Optimization & Gradients · Chapter 2

The gradient in multiple dimensions

Hook

You know how to find downhill when there's one direction to move in. What does "downhill" even mean once there's an xx and a yy?

Intuition

This is a bowl-shaped landscape seen from above — brighter means lower. Drag the point around. There's no single tangent line anymore, but there's still an arrow pointing the steepest way out.

Formalize

That arrow is the gradient — now a vector, one component per input:

∇f(x,y)=(∂f∂x, ∂f∂y)\nabla f(x,y) = \left(\frac{\partial f}{\partial x},\ \frac{\partial f}{\partial y}\right)
  • ∇f(x,y)\nabla f(x,y) — the gradient vector at the point (x,y)(x,y).
  • (x,y)(x,y) — the point in the plane where the gradient is evaluated.
  • ∂f/∂x\partial f/\partial x — the partial derivative of ff with respect to xx, holding yy fixed.
  • ∂f/∂y\partial f/\partial y — the partial derivative of ff with respect to yy, holding xx fixed.
  1. Hold the other variable fixed

    Each partial derivative is computed exactly like Chapter 1's derivative — just holding the other variable fixed.

  2. Work it out for f(x,y) = x² + y²

    Treating yy as a constant, ∂f/∂x=2x\partial f/\partial x = 2x; treating xx as a constant, ∂f/∂y=2y\partial f/\partial y = 2y.

Play
∇f = (8.0, -6.0), |∇f| = 10.0

Drag the point and watch both components change together, and the arrow's length track the gradient's magnitude — longer means steeper.

Worked example
  1. Compute each partial derivative at (3, 4)
    ∇f=(2×3, 2×4)=(6,8)\nabla f = (2 \times 3,\ 2 \times 4) = (6, 8)
  2. Compute the gradient's magnitude
    ∣∇f∣=62+82=100=10|\nabla f| = \sqrt{6^2+8^2} = \sqrt{100} = 10

    The same 3:4:5 triangle from the vectors chapter, showing up again.

Checkpoint

Drag the point until ∇f reads (approximately) (0, 0) — the flat spot at the bottom of the bowl.

∇f = (8.0, -6.0)
Drag the point to try it
Summary
∇f(x,y)=(∂f∂x, ∂f∂y)\nabla f(x,y) = \left(\frac{\partial f}{\partial x},\ \frac{\partial f}{\partial y}\right)

In multiple dimensions, the gradient is a vector of partial derivatives — one per input — and it still points in the direction of steepest increase, zeroing out exactly at a flat spot like a minimum.