The interpretability–accuracy tradeoff

Same ten points, two models. The "simple" model is one rule — one threshold, two regions — and you could recite its entire logic in a sentence. The "complex" model has enough boundaries to get every point right, but describing why it drew each one starts to require pointing at the data itself, not stating a rule.

For a set of decision regions, accuracy is just the fraction correctly classified. A simple, decreasing proxy for interpretability is one over the number of regions the model uses:

• accuracy — the fraction of the dataset the model classifies correctly.
• correct — the number of points the model classifies correctly.
• $n$ — the total number of points in the dataset.
• interpretability — a simple, decreasing proxy for how easy the model is to state as a short list of rules.
• regions — the number of decision regions the model carves the input space into.

1. What the proxy stands in for

   This interpretability score is a stand-in for something real: a model built from a handful of regions is one a person can hold in their head and restate as a short list of rules.

2. The tradeoff it makes visible

   A model built from dozens of regions — one for nearly every data point — has traded that away for the ability to fit exceptions no simple rule ever could.

Step through all three models. Each added split fixes one more of the dataset's exceptions — but every fix also grows the rule. By the third model, every single point is classified correctly, at the cost of a rule with four separate regions instead of two.

On the same 10-point dataset:

1. Stump — 1 split, 2 regions

   Accuracy $7/10 = 70\%$. Interpretability $1/2 = 0.50$. It gets the three "noisy" points near $x=7,8,9$ wrong, because a single threshold can't carve out a middle exception.

2. Small tree — 2 splits, 3 regions

   Accuracy $9/10 = 90\%$. Interpretability $1/3 \approx 0.33$. The extra split recovers $x=7,8,9$, but now misses the lone point at $x=10$.

3. Black box — 3 splits, 4 regions

   Accuracy $10/10 = 100\%$. Interpretability $1/4 = 0.25$. A third split finally isolates $x=10$ too — every point correct, at the cost of the most regions of the three.