Prompting & in-context learning

A handful of demonstrated (input, output) pairs, and a new query. The query attends to the demonstrations by how close their inputs are to its own — exactly Part IV's attention mechanism — and blends their outputs accordingly. Swap the demonstrations, and the same frozen mechanism gives a different answer.

In-context learning treats the demonstrations in a prompt as an implicit, tiny training set, processed entirely within a single forward pass — no gradient step, no weight update:

• $x_q$ — the query input: the thing actually being asked about.
• $x_i$ — the input of the $i$-th demonstration example shown in the prompt.
• $y_i$ — the output demonstrated alongside $x_i$.
• $\hat{y}(x_q)$ — the model's predicted output for the query, blended from the demonstrations.
• $K(x_q, x_i)$ — a similarity kernel: how close the query is to a given demonstration's input.
• $\sigma$ — the kernel's bandwidth, controlling how fast similarity falls off with distance.
• $i, j$ — indices ranging over the demonstration examples in the prompt.

1. Attention as kernel regression

   This is a kernel-regression view of what attention over demonstrations computes: weight each example by similarity to the query, then blend their answers.

2. Weights depend only on inputs, never on outputs

   The weights depend only on the inputs $x_i$ — completely independent of what pattern the outputs $y_i$ happen to encode.

Both prompts share identical demonstration inputs ($1, 2, 3$), so both produce the exact same attention weights for the same query. The only thing that differs is which outputs were demonstrated — and that alone flips the sign of the final answer.

Query $x_q = 2.5$, two different sets of demonstrations at the same three input values:

1. The weights come from the inputs alone

   Both prompts use $x \in \{1, 2, 3\}$ against query $x_q = 2.5$, with bandwidth $\sigma=1$:

   • $K(2.5,1) = \exp(-(2.5-1)^2/2) = \exp(-1.125) \approx 0.325$
   • $K(2.5,2) = \exp(-(2.5-2)^2/2) = \exp(-0.125) \approx 0.882$
   • $K(2.5,3) = \exp(-(2.5-3)^2/2) = \exp(-0.125) \approx 0.882$
   • Sum $\approx 0.325+0.882+0.882 = 2.090$
   • Normalized weights:
     • $0.325/2.090\approx0.155$
     • $0.882/2.090\approx0.422$
     • $0.882/2.090\approx0.422$

   Both prompts produce weights $\approx[0.155, 0.422, 0.422]$ — identical, because the kernel only ever looks at how far $x_q$ is from each $x_i$.

2. 'Double' demonstrations blend toward roughly double

   With $y = \{2, 4, 6\}$ demonstrated, the weighted blend is $0.155(2) + 0.422(4) + 0.422(6) \approx 0.311 + 1.689 + 2.534 = 4.534$ — close to the true doubled value of $5$, though not exact, since a weighted average can only interpolate, not perfectly extrapolate a slope.

3. 'Negate' demonstrations, same weights, opposite answer

   With $y = \{-1, -2, -3\}$ demonstrated instead, the identical weights blend to $0.155(-1) + 0.422(-2) + 0.422(-3) \approx -0.155 - 0.845 - 1.267 = -2.267$ — close to the true negated value of $-2.5$, and of the opposite sign entirely from the "double" case.