Part IX built precision, recall, and F1 from one confusion matrix. Compute the same quantities separately for two different groups, and a new question appears that a single confusion matrix never had to answer: should those numbers be equal across groups — and equal by which definition?
Switch fairness definitions. The bars move differently depending on which one you're looking at — this isn't one classifier being "somewhat unfair"; it's three genuinely different questions, each with its own answer.
Three individually reasonable definitions of "fair," computed per group from the exact same confusion matrix counts as Part IX:
- — true positives: people the classifier correctly predicted positive, within that group.
- — false positives: people the classifier incorrectly predicted positive, within that group.
- — false negatives: actual positives the classifier missed, within that group.
- — the total number of people in that group.
- Equal treatment breaks equal selection
When the two groups have different base rates — different actual fractions of positives — a classifier that's calibrated the same way for both groups (equal opportunity, equal precision) will not select both groups at the same rate.
- Equal selection breaks equal treatment
Symmetrically, a classifier that selects both groups equally (demographic parity) will not have matching precision or recall across them.
- This is a proven impossibility, not a bug
This isn't a modeling bug to fix; it's a mathematical fact about what happens when base rates genuinely differ, proven formally by Kleinberg, Mullainathan & Raghavan and by Chouldechova in 2016.
Three gaps, three different sizes, none of them zero. A classifier "fixed" to close one of these gaps exactly would, in general, open up the other two further — that's the impossibility result, not a failure of this particular classifier.
Group A (base rate 0.5) and Group B (base rate 0.2), same classifier:
- Demographic parity
- Group A: selected
- Group B: selected
- Gap:
- Equal opportunity (recall)
- Group A:
- Group B:
- Gap: — B's actual positives are easier to catch, purely because there are so few of them.
- Predictive parity (precision)
- Group A:
- Group B:
- Gap: — the largest of the three, and the one a naive "check the accuracy" audit would most likely miss entirely.
Find the fairness metric, among the three, that shows the largest disparity between the two groups.
There's no purely mathematical answer to which of these three definitions is "the" right one — that's a policy question about what kind of error matters more in a given context (a missed loan approval versus an unjustified denial, say), not something an auditor can resolve by computing harder. What an audit can do is what this chapter did: compute all the reasonable definitions, on the real base rates, and make the disagreement visible instead of picking one metric and calling the system fair. The next chapter turns from auditing a model's outputs to looking inside it — decomposing what a single neuron is actually doing.