Consistency models

Drag $t$ anywhere. The second point never moves — every value of $t$ on this trajectory maps to the exact same endpoint, through one function call, no matter how far from $t{=}1$ you start.

A consistency model learns a function $f(x_t, t)$ trained so that any two points on the same trajectory map to the same output — the endpoint. On this chapter's straight-line paths, that function has a closed form:

• $f(x_t, t)$ — the consistency function: maps any point on a trajectory, at any time $t$, straight to that trajectory's endpoint.
• $x_t$ — the point on the trajectory at time $t$.
• $x_0$ — the trajectory's starting point (noise, at $t=0$).
• $x_1$ — the trajectory's endpoint (data, at $t=1$) that $f$ should output for every $t$ on this trajectory.
• $t$ — the time parameter identifying where along the trajectory the input point sits.

1. Self-consistency holds for any t

   The middle expression above holds for any $t$ — $f$ doesn't care how far along the trajectory it's asked to evaluate, it always outputs the endpoint.

2. Curved trajectories have no closed form

   Real consistency models are trained on the genuinely curved trajectories that diffusion models produce, where no such closed form exists.

3. A distillation loss teaches the shortcut

   A consistency-distillation loss explicitly penalizes disagreement between $f$'s outputs at two nearby points on the same trajectory, teaching the shortcut a curved path doesn't hand you for free.

Five wildly different starting points along the same trajectory — five identical bars. That flatness is the trained property, not a coincidence of the numbers: a consistency model that got this wrong would show five different heights instead.

Trajectory $(x_0, x_1) = (-2, 5)$, checked at two very different values of $t$:

1. t = 0.3

   $x_{0.3} = 0.7({-}2) + 0.3(5) = 0.1$. $f(x_{0.3}, 0.3) = 0.1 + 0.7(7) = 5$.

2. t = 0.8

   $x_{0.8} = 0.2({-}2) + 0.8(5) = 3.6$. $f(x_{0.8}, 0.8) = 3.6 + 0.2(7) = 5$.

3. Same trajectory, same answer

   Two completely different inputs — $(0.1, 0.3)$ and $(3.6, 0.8)$ — produce the identical output. That's not a special property of $t{=}0.3$ or $t{=}0.8$; it holds for every $t$ on this trajectory, because both points sit on the same straight line to $x_1=5$.