Part XVII — LLM Post-Training: SFT, DPO, GRPO & Reasoning / Test-Time Compute · Chapter 1

The modern LLM post-training pipeline

Hook

This part has three chapters left to teach before it's done: instruction tuning, preference optimization, and RL against verifiable rewards. It's tempting to treat them as three competing ideas. They're not — they're three stages of one pipeline, each one only possible because the previous one already ran.

Intuition

Step forward through the pipeline, one stage at a time, on one held-out benchmark. Nothing here jumps straight from a raw pretrained model to a fully aligned reasoning model — every stage takes whatever the previous stage produced as its starting point.

Formalize

Each stage's output accuracy is whatever the previous stage achieved, plus that stage's own contribution:

acci=acci1+Δi\text{acc}_i = \text{acc}_{i-1} + \Delta_i
  • acci\text{acc}_i — benchmark accuracy after stage ii has run.
  • acci1\text{acc}_{i-1} — the previous stage's output; for stage 11 (SFT), this is the raw pretrained model's accuracy.
  • Δi\Delta_i — stage ii's own marginal contribution, on top of whatever it started from.
  1. Stage 1 — SFT teaches the model to attempt an answer at all

    Chapter 2's masked cross-entropy loss turns a text-completer into something that at least tries to follow an instruction.

  2. Stage 2 — preference optimization picks the better of the attempts it can already make

    RLHF or DPO (Chapters 3-4, and the KTO/SimPO variants) refine which SFT-style answer the model prefers to produce — no new capability, just better taste among outputs it could already generate.

  3. Stage 3 — RLVR pushes past what any human preference alone could teach

    On tasks with a checkable ground truth, RLVR (this part's next chapters) optimizes directly against verifiable correctness — a signal preference data never had access to.

Play

Three bars, three completely different mechanisms, three different-sized contributions. SFT does the heaviest lifting here — going from "can't follow instructions at all" to "follows them passably" is a bigger jump than any later polish. That doesn't make the later stages optional: each is solving a problem the previous one structurally can't.

Worked example

A pretrained base model scoring 0.150.15 on some held-out benchmark, run through all three stages:

  1. After SFT

    Accuracy rises to 0.450.45 — a gain of 0.450.15=0.300.45 - 0.15 = 0.30, the largest single jump in the pipeline.

  2. After preference optimization

    Accuracy rises to 0.620.62 — a gain of 0.620.45=0.170.62 - 0.45 = 0.17.

  3. After RLVR

    Accuracy rises to 0.810.81 — a gain of 0.810.62=0.190.81 - 0.62 = 0.19.

  4. Total improvement

    0.810.15=0.660.81 - 0.15 = 0.66, exactly the sum of the three individual gains: 0.30+0.17+0.19=0.660.30+0.17+0.19=0.66. Every stage's contribution is additive and separately measurable — none of them do each other's job.

Checkpoint

Find the stage whose own marginal accuracy gain is the largest of the three.

Pick a stage to try it
Summary
acci=acci1+Δi\text{acc}_i = \text{acc}_{i-1} + \Delta_i

"Post-training" isn't one algorithm competing with the others for the title of best technique — it's a pipeline where SFT, preference optimization, and RLVR each solve a problem structurally out of reach for the others: attempting an answer at all, picking the better of two attempts, and optimizing against ground truth where one exists. The rest of this part builds each stage's machinery in turn, starting with why a learned reward model — not yet a verifiable one — was the first way anyone taught a language model what "better" even means.