Part IV — Peer-to-Peer & Decentralized Data · Chapter 6

Dynamo-style storage

Hook

Amazon's shopping cart must accept a write even when part of the network is unreachable — a customer adding an item should never be told "no." But if two datacenters each accept a concurrent write to the same cart while cut off from each other, how does the store even know they conflict, let alone reconcile them later?

Intuition
Node A's clock{}
equal
Node B's clock{}

Click "A writes" once, then "B writes" once, without ever syncing — the verdict reads concurrent: neither clock has seen the other's update, so neither can claim to be newer. Now hit "Sync" and write on A again: the verdict flips to a clean after, because A's new clock has now seen everything B had.

Formalize

A vector clock gives every replica its own counter, incremented only on that replica's own writes. Comparing two clocks aa and bb coordinate-by-coordinate tells you their causal relationship:

ab    i, aibi  and  i, ai<bia \prec b \iff \forall i,\ a_i \le b_i \ \text{ and } \ \exists i,\ a_i < b_i

If neither aba \prec b nor bab \prec a holds, the two writes are concurrent — a genuine conflict, not just a stale read. Separately, Dynamo's sloppy quorum guarantees any read quorum of size RR and write quorum of size WW, out of NN replicas, must share at least one node:

R+W>NR + W > N
  • aia_i — replica ii's counter as recorded in vector clock aa.
  • aba \prec b — "aa happened-before bb": every counter in aa is \le the matching one in bb, with at least one strictly less.
  • RR, WW, NN — read quorum size, write quorum size, and total replicas in the preference list.
  1. Pigeonhole guarantees the overlap

    If a size-RR read set and a size-WW write set were completely disjoint, together they'd only need R+WR+W distinct nodes. The moment R+W>NR+W>N, there aren't enough distinct nodes left for that — some node has to be in both.

  2. Hinted handoff keeps writes flowing around a down node

    When a node in the write quorum is unreachable, the write goes to the next healthy node in the preference list instead, which holds a hint: "forward this to the down node once it's back." The write still succeeds; nothing is silently dropped.

Play
W=3 targets when N2 is healthy: N1, N2, N3

Slide R from 1 up to 5 and watch the overlap column: below R=3 it can hit zero, at R=3 and above it always includes at least N3. Then flip N2 to "down" and watch the W=3 write targets skip straight past it to N4, with N3 now shown holding a hint for N2 — the write never blocks on the down node.

Worked example

A 5-node preference list N1..N5, write quorum W=3={W=3=\{N1, N2, N3}\}:

  1. Two concurrent writes, then a causal one
    1. Node A writes with an empty clock: {A:1}\{A{:}1\}
    2. Node B writes independently: {B:1}\{B{:}1\} — concurrent with A's write, since neither {A:1}{B:1}\{A{:}1\} \prec \{B{:}1\} nor {B:1}{A:1}\{B{:}1\} \prec \{A{:}1\} holds
    3. After a sync, A writes again: {A:2,B:1}\{A{:}2, B{:}1\}, which is now strictly after B's {B:1}\{B{:}1\}, since every one of B's counters (B:1B{:}1) is matched or exceeded
  2. R=2 is not enough

    Read set {\{N4, N5}\} against write set {\{N1, N2, N3}\}: no overlap at all. R+W=2+3=5R+W=2+3=5, which is not strictly greater than N=5N=5 — exactly the boundary case where a gap is still possible.

  3. R=3 closes the gap

    Read set {\{N3, N4, N5}\} against the same write set: they share N3. R+W=3+3=6>5R+W=3+3=6>5, so this overlap isn't a coincidence — every size-3 read set must hit the size-3 write set.

  4. Hinted handoff around N2

    Walking the preference list N1..N5 to fill 3 slots:

    • N1: healthy → included (slot 1)
    • N2: down → skipped; its hint attaches to the next healthy node reached
    • N3: healthy → included (slot 2), and holds N2's hint
    • N4: healthy → included (slot 3)

    Write quorum becomes N1, N3, N4 — and N3, the node that stepped in right after the gap, is recorded as holding N2's hint.

Checkpoint

With the write quorum fixed at W=3 out of N=5, slide the read quorum R to the exact minimum value that guarantees every possible read overlaps every possible write — not just any R past that point.

Drag the R slider
Summary
R+W>Nguarantees overlap,ab    i, aibi (i, ai<bi)R + W > N \quad\text{guarantees overlap}, \qquad a \prec b \iff \forall i,\ a_i \le b_i \ (\exists i,\ a_i < b_i)

Dynamo's whole design is "never refuse a write": sloppy quorums keep writes and reads overlapping often enough to stay useful without demanding every replica be reachable, hinted handoff keeps a write succeeding around a temporarily down node, and vector clocks are the only honest way to tell a genuine conflict apart from a stale copy once those concurrent writes need reconciling. The capstone puts these same vector clocks to work again, this time reconciling a write made mid-flight through a gossip-based membership change.