Integrated gradients

Instead of one gradient at one point, walk from a baseline to the input in small steps and average the gradient along the way. More steps means a finer walk along the exact same path, not a different destination — and the result converges to something a single gradient could never see.

Integrated gradients attribute a prediction to an input by integrating the gradient along the straight-line path from a baseline to the actual input:

• $\text{IG}(x)$ — the integrated gradients attribution for input $x$: how much of the output change gets credited to this feature.
• $x$ — the actual input value being explained.
• $x_{\text{baseline}}$ — the reference input (often all zeros) that the path starts from, representing "no information" for that feature.
• $\alpha$ — the interpolation fraction along the straight-line path from baseline to input, ranging from $0$ to $1$.
• $f$ — the model, as a function of its input.

1. Approximated by a Riemann sum

   In practice this integral is approximated by a Riemann sum over $m$ steps along that path.

2. Completeness axiom

   As $m$ grows, the attribution converges to exactly $f(x) - f(x_{\text{baseline}})$, the true change in output — the key guarantee behind this method.

3. Saliency is the degenerate case, m=1

   Plain saliency is just this method with a single step: the gradient at the input alone, with no path at all.

The model curve is flat near both ends and steep only in the middle. The path from baseline to input crosses that entire steep region — but the gradient at the input alone sits in the flat, saturated tail, where it's almost meaningless.

$f(x) = \sigma(5x)$, baseline $x_0=0$, input $x=2$:

1. The true output change is large, the gradient at the input is tiny

   • $f(0)=\sigma(0)=0.5$
   • $f(2)=\sigma(10)=1/(1+e^{-10})\approx0.99995$

   A change of nearly $0.5$. The derivative $f'(x)=5\sigma(5x)(1-\sigma(5x))$ at $x=2$: $f'(2)=5\times0.99995\times(1-0.99995) \approx5\times0.99995\times0.00005\approx0.000227$ — the input sits so deep in saturation that its own gradient badly understates how much it mattered.

2. One step reduces to the same blind spot

   With only 1 integration step, IG evaluates the gradient only at the input itself — identical to plain saliency. The result, $\approx0.00045$, misses over 99.9% of the true $\approx0.4999546$ output change.

3. More steps recover the missing attribution

   • At 10 steps: $\approx0.375$ (gap $\approx0.125$)
   • At 50 steps: $\approx0.475$ (gap $\approx0.025$)
   • At 500 steps: $\approx0.497$ (gap $\approx0.0025$)

   Steadily converging toward the exact true change, because most of that path crosses the steep, high-gradient middle that a single endpoint gradient never sees.