LIME

The curve is a "black box" — LIME — short for Local Interpretable Model-agnostic Explanations — never sees its formula, only its outputs. Drag the query point $x_0$: LIME samples a few points near it, weights the close ones more than the far ones, and fits the simplest possible model — a straight line — through just that weighted neighborhood.

Around a query point $x_0$, take samples $x_i$ nearby, weight each by a kernel that favors closeness, and fit a weighted linear regression:

• $x_0$ — the query point: the specific input whose prediction is being explained.
• $x_i$ — one of the sample points drawn near $x_0$.
• $w_i$ — the weight given to sample $x_i$, which shrinks the farther $x_i$ is from $x_0$.
• $\sigma$ — the kernel width, controlling how quickly the weight falls off with distance.
• $\hat y$ — the local linear model's predicted output.
• $\text{slope}, \text{intercept}$ — the two parameters of the local straight-line fit.

1. A weighted least-squares fit

   The slope and intercept minimize $\sum_i w_i(y_i - \hat y_i)^2$ — an ordinary least-squares fit, just with each point's error scaled by how close it is to $x_0$.

2. Model-agnostic by construction

   Nothing here needs the black box's internals: LIME only ever calls it as a function, exactly like a saliency map does, but a linear fit instead of a per-input gradient.

Drag $x_0$ anywhere and compare LIME's local slope to the curve's actual derivative at that point, $2x_0$ — a fact LIME itself never gets to see, since it only ever queries the black box, never its formula. Watch how closely a local, linear approximation can still track a genuinely curved function, as long as it's never asked to explain anywhere but right around $x_0$.

The black box computes $f(x)=x^2$ (unknown to LIME). At $x_0=3$, with kernel width $\sigma=1.5$, LIME samples $x \in \{1,2,3,4,5\}$, giving $y \in \{1,4,9,16,25\}$:

1. Fit the weighted local line

   The Gaussian kernel weight for each offset $dx$, $w=\exp(-dx^2/4.5)$:

   • $dx=-2$: $w\approx0.411$
   • $dx=-1$: $w\approx0.801$
   • $dx=0$: $w=1$
   • $dx=1$: $w\approx0.801$
   • $dx=2$: $w\approx0.411$

   These are symmetric around $x_0=3$, so the weighted mean of $x$ is exactly $\bar x=3$, and the weighted mean of $y$ comes out to $\bar y\approx10.43$ (pulled above $f(3)=9$ by the two farthest, largest-$y$ samples). Plugging every sample's $(x_i-\bar x, y_i-\bar y)$ pair into the weighted least-squares slope formula, $\sum w_i(x_i-\bar x)(y_i-\bar y) \big/ \sum w_i(x_i-\bar x)^2$, gives slope $=6$; the intercept then follows from $\bar y - \text{slope}\cdot\bar x \approx 10.43-6(3) \approx-7.57$.

2. Compare to the true derivative

   $f'(x) = 2x$, so $f'(3)=6$ — LIME's local slope matches the exact derivative, even though LIME never computed a derivative at all, only fit a line through five sampled points.

3. But the line doesn't pass through (3, 9)

   At $x=3$ the fitted line gives $6(3)-7.57 \approx 10.43$, not $9$. LIME's local model tracks the slope faithfully — that's what makes it a good explanation of sensitivity — but it's a regression through a neighborhood, not an interpolation pinned to the exact point.