Part III — Probability, Information Theory & Bayesian Inference · Chapter 4

Conditional probability & Bayes' rule

Hook

A test comes back positive. How likely is it that you actually have the condition — really?

Intuition
Has it, tests positive Doesn’t, tests positive

Each square is one of 100 people. Teal means they have the condition and tested positive; ochre means they don't have it but tested positive anyway. Drag the base rate down — watch the ochre squares start to outnumber the teal ones, even though the test itself hasn't changed at all.

Formalize

P(AB)P(A\mid B) reads "the probability of AA, given that BB happened" — your belief in AA updated by evidence BB. Bayes' rule says how to compute it —

P(AB)=P(BA)P(A)P(B)P(A\mid B) = \frac{P(B\mid A)\,P(A)}{P(B)}
  • AA — the event we want to know about (e.g., "has the condition").
  • BB — the observed evidence (e.g., "tested positive").
  • P(AB)P(A\mid B) — the updated probability of AA given that BB happened (the posterior).
  • P(BA)P(B\mid A) — the probability of seeing evidence BB if AA is true (the likelihood, e.g. sensitivity).
  • P(A)P(A) — the probability of AA before seeing any evidence (the prior, or base rate).
  • P(B)P(B) — the overall probability of observing BB, across all cases.
  1. Map A and B to the demo

    Here AA is "has the condition," BB is "tested positive."

  2. Read the likelihood and the prior

    P(BA)P(B\mid A) is the test's sensitivity — how often it catches a real case. P(A)P(A) is the base rate you just dragged.

Play
Has it, tests positive Doesn’t, tests positive
P(condition | positive) = 32.1%

Watch P(AB)P(A\mid B) track the base rate: the test's accuracy never changes, yet how much a positive result should actually convince you depends heavily on how rare the condition was to begin with.

Worked example

Base rate 1%1\%, sensitivity 90%90\%, false-positive rate 10%10\%:

  1. Find the overall chance of a positive test
    P(B)=0.9(0.01)+0.1(0.99)=0.009+0.099=0.108P(B) = 0.9(0.01) + 0.1(0.99) = 0.009 + 0.099 = 0.108
  2. Apply Bayes' rule
    P(AB)=0.9×0.010.1088.3%P(A\mid B) = \frac{0.9 \times 0.01}{0.108} \approx 8.3\%
  3. Read what it means

    A "90% accurate" test, and a positive result still means less than a 1-in-10 chance of actually having it — because the condition was rare enough that false alarms from the other 99% swamp the true positives.

Checkpoint

Drag the base rate until P(condition | positive) reads (approximately) 50%.

Has it, tests positive Doesn’t, tests positive
P(condition | positive) = 32.1%
Drag the slider to try it
Summary
P(AB)=P(BA)P(A)P(B)P(A\mid B) = \frac{P(B\mid A)\,P(A)}{P(B)}

A test's own accuracy isn't the whole story — how rare the thing you're testing for is changes what a positive result actually means. Bayes' rule is the exact accounting for that.