Adversarial attacks & robustness (FGSM & PGD)

Hook

A model can be perfectly accurate on every image you'd ever show it and still fail catastrophically on one an attacker built on purpose. The failure doesn't need a strange-looking input — it needs a change too small to see.

Intuition

input, each pixel nudged by 0.00

P(positive class)

0.7109

still classified positive

epsilon0.00

Drag epsilon. Every one of the 9 pixels shifts by the exact same tiny amount — nothing in the grid looks obviously different — yet the confidence score falls steadily, and eventually crosses the decision boundary entirely.

Formalize

FGSM (Fast Gradient Sign Method) perturbs every input dimension by a fixed budget $\epsilon$ , in whichever direction most decreases the classifier's score — for a linear model, that direction is just the negative sign of each weight:

x'_i = x_i - \epsilon \cdot \text{sign}(w_i), \qquad \Delta(w \cdot x) = -\epsilon \sum_i |w_i|

$x_i$ — the original value of the $i$ -th input dimension (one pixel).
$x'_i$ — that same dimension's value after the perturbation is applied.
$\epsilon$ — the fixed perturbation budget applied to every dimension.
$w_i$ — the classifier's weight on dimension $i$ .
$\Delta(w \cdot x)$ — the resulting shift in the classifier's weighted sum (its score) once every dimension has been nudged.

Each dimension's change is imperceptibly small
Every individual $|x'_i - x_i|$ is bounded by $\epsilon$ — small, imperceptible on its own.
The total shift scales with dimension count, not epsilon alone
The shift in the dot product the classifier actually uses scales with the number of dimensions. A perturbation invisible in any single pixel becomes a large, deliberate shift once summed across all of them.

Play

ε = 0.05

0.6106

ε = 0.1

0.5000

ε = 0.15

0.3894

ε = 0.2

0.2891

each pixel moves by at most 0.2 — but the dot product moves by 9× that, since all 9 dimensions shift together

Confidence falls steadily as epsilon grows — not a single dramatic jump, a gradual decline that eventually crosses 0.5. The classifier isn't confused about any individual pixel; it's just adding up nine small, coordinated pushes in exactly the wrong direction.

Worked example

9 pixels at 0.6, weights all 1, bias $-4.5$ :

Before any perturbation
$w \cdot x + b = 9(0.6) - 4.5 = 0.9$ . $\sigma(0.9) = \frac{1}{1+e^{-0.9}}$ : $e^{-0.9}\approx0.4066$ , so $\sigma(0.9)\approx\frac{1}{1.4066}\approx0.711$ — confidently positive.
At epsilon = 0.1
Every pixel drops to $0.5$ . $w \cdot x + b = 9(0.5) - 4.5 = 0$ . $\sigma(0) = 1/(1+e^0) = 1/2 = 0.5$ — sitting exactly on the boundary.
At epsilon = 0.15
Every pixel drops to $0.45$ — barely different from $0.5$ . $w \cdot x + b = 9(0.45) - 4.5 = -0.45$ . $\sigma(-0.45) = \frac{1}{1+e^{0.45}}$ : $e^{0.45}\approx1.568$ , so $\sigma(-0.45)\approx\frac{1}{2.568}\approx0.389$ — now confidently classified in the other class.

Checkpoint

Find the smallest epsilon, among the candidates, that actually flips the classification.

Pick an epsilon to try it

Summary

x' = x - \epsilon \cdot \text{sign}(w), \qquad \Delta(w\cdot x) = -\epsilon\sum_i |w_i|

This chapter's classifier was linear and its weights were visible — real adversarial attacks on deep networks use the same sign-of-gradient idea, computed via backpropagation through the whole network instead of read directly off a weight vector, which is exactly why adversarial examples generalize across architectures the attacker never even saw. Robustness training (adding perturbed examples like these directly into the training set) is one real defense; it trades some clean accuracy for resistance to exactly this kind of engineered nudge. The next chapter looks at a related but distinct failure: not a classifier fooled by pixels, but a language model talked past its own safety training by phrasing alone.