A weight matrix stored as 32-bit floats moves through memory as 32-bit floats, every time it's used. Store it in 4 bits instead and there's four times less to move — but 4 bits only has 16 distinct values, so most weights can no longer be represented exactly. The question is how much that costs, and whether it's worth paying.
-1.6 ÷ scale (0.40) → level -4 → reconstructed -1.60 — exact, zero error
Eight FP32 weights, one shared scale. Pick each one and watch it divide by the scale, round to the nearest integer level, and multiply back — sometimes landing exactly where it started, sometimes a little off.
Post-training quantization (the family AWQ, GPTQ, and GGUF all belong to) picks one scale per group of weights, then stores each weight as a small signed integer level:
- — the original FP32 weight.
- — the shared scale for this group of weights.
- — the weight's rounded 4-bit integer level, one shared FP32 scale away from being a real number again.
- — the dequantized weight actually used at inference time.
A weight only survives round-tripping exactly when it was already a multiple of — everything else rounds to the nearest representable level and keeps the difference as error:
- The scale sets what's exact
Levels only cover for 4-bit — 16 evenly spaced values. Any weight in between rounds to whichever is closer.
- Compression comes from bit width, not from the scale
Sixteen weights at 4 bits packed two-per-byte, plus one shared 32-bit scale for the whole group, is close to a flat 8× smaller than the same sixteen weights at 32 bits each — the smaller the group, the more that one shared scale's overhead matters.
The same eight weights, both ways: four bytes each as FP32, versus packed 4-bit levels plus one shared scale. The compression is exact and predictable; the error is the price, and it's exactly as large as the rounding above made it — no more, no less.
- The scale
Largest-magnitude weight is ; the top 4-bit level is 7. Scale .
- Most weights round exactly
- exactly — level , dequantized back to
- exactly
- exactly
- exactly
- exactly
- exactly
All six land exactly on a level, zero error.
- Two weights don't
- , rounds to level , dequantizes to — off by
- , rounds to level , dequantizes to — off by
It's and that land off-grid, each by exactly , giving a mean squared error of across all eight.
A fresh weight, 1.1, needs to be quantized with the same scale (0.40). Which 4-bit level does it round to?
AWQ, GPTQ, and GGUF all sit on top of this same round-trip — they differ in which weights they protect from rounding error. AWQ picks the scale per group to protect the small fraction of "salient" weights activations actually depend on most; GPTQ solves for the rounding order that minimizes the accumulated error layer by layer; GGUF is a storage format for these same integer-plus-scale representations, tuned for fast CPU and mixed-precision inference. All of them are managing the exact trade this chapter's formula makes visible: memory saved is proportional to bits saved, and error is exactly what's left after rounding to the nearest representable level. The next chapter puts a quantized model into an actual serving loop, where the KV-cache — not the weights — becomes the memory bottleneck.