Denoising Diffusion Probabilistic Models (DDPM)

The forward process (top row) is fixed and requires no learning at all — just mix in a little more noise at every step, according to a fixed schedule, until the data is almost unrecognizable. The reverse process (bottom row) undoes it one step at a time. With a perfect denoiser, reversing recovers the original value exactly — the same numbers, in reverse order.

Forward, adding noise at step $t$: $x_t = \sqrt{\alpha_t}\, x_{t-1} + \sqrt{\beta_t}\, \epsilon_t$, where $\beta_t$ is how much noise this step adds and $\alpha_t = 1-\beta_t$. Reverse, removing it, given a predicted noise $\hat\epsilon_t$ from a trained model:

• $x_t$ — the (increasingly noisy) value at step $t$.
• $x_{t-1}$ — the value one step earlier: less noisy, one step closer to the original data.
• $\beta_t$ — how much noise step $t$ adds to the signal, fixed by a schedule.
• $\alpha_t$ — shorthand for $1-\beta_t$: how much of the previous signal survives this step.
• $\hat\epsilon_t$ — the trained model's predicted noise at step $t$, used to undo the forward step.

1. Reverse is the forward equation, solved backward

   This is exactly the forward equation solved for $x_{t-1}$.

2. A perfect predictor exactly inverts it

   If $\hat\epsilon_t$ equals the true $\epsilon_t$ at every step, this exactly inverts the forward process.

3. Training reduces to one question

   The entire training problem reduces to one question: can a network predict, from a noisy $x_t$, what noise was added to get there?

Drag the denoiser's quality down from perfect (1) toward clueless (0, always predicting zero noise). The reconstructed value drifts further from the true $x_0$ — but even at quality $0$, it lands closer to $5$ than the pure-noise starting point did, because the $\sqrt{\alpha_t}$ scaling alone pulls values back toward a reasonable range at every step. A real diffusion model's whole job is training that noise predictor well enough to close this gap.

1. Forward — destroy the signal

   With noise schedule $\beta=[0.1,0.2,0.3,0.4]$ (so $\alpha=[0.9,0.8,0.7,0.6]$) and fixed noise samples $\epsilon=[1,-0.5,0.8,-1.2]$:

   1. $x_1 = \sqrt{0.9}(5) + \sqrt{0.1}(1) \approx 4.743+0.316 = 5.06$
   2. $x_2 = \sqrt{0.8}(5.06) + \sqrt{0.2}(-0.5) \approx 4.526-0.224 = 4.30$
   3. $x_3 = \sqrt{0.7}(4.30) + \sqrt{0.3}(0.8) \approx 3.600+0.438 = 4.04$
   4. $x_4 = \sqrt{0.6}(4.04) + \sqrt{0.4}(-1.2) \approx 3.128-0.759 = 2.37$
2. Reverse with a perfect denoiser

   Running the reverse formula with $\hat\epsilon_t$ set to the exact same true noise, from $x_4=2.37$ backward:

   1. $x_3 = (2.37 - \sqrt{0.4}(-1.2))/\sqrt{0.6} \approx (2.37+0.759)/0.775 = 4.04$
   2. $x_2 = (4.04 - \sqrt{0.3}(0.8))/\sqrt{0.7} \approx (4.04-0.438)/0.837 = 4.30$
   3. $x_1 = (4.30 - \sqrt{0.2}(-0.5))/\sqrt{0.8} \approx (4.30+0.224)/0.894 = 5.06$
   4. $x_0 = (5.06 - \sqrt{0.1}(1))/\sqrt{0.9} \approx (5.06-0.316)/0.949 = 5.00$

   Exactly retraces those numbers backward.

3. Reverse with a useless denoiser

   Same formula, but every $\hat\epsilon_t=0$, from $x_4=2.37$ backward:

   1. $x_3 = (2.37-0)/\sqrt{0.6} \approx 3.06$
   2. $x_2 = (3.06-0)/\sqrt{0.7} \approx 3.66$
   3. $x_1 = (3.66-0)/\sqrt{0.8} \approx 4.09$
   4. $x_0 = (4.09-0)/\sqrt{0.9} \approx 4.31$

   The right neighborhood, but nowhere near exact, because every step's error compounds into the next.