Tree-based feature importance

Hook

Integrated gradients and PDPs both probe an already-trained model from the outside, after the fact. A decision tree already knows exactly which feature did the most work — it decided that at every single split, while training. Reading it back out costs nothing extra.

Intuition

x1 @ threshold 4.5

0.5488

n=8 samples reach this node

Step through each node in a small tree and see which feature it picked, and how much impurity that split actually removed. The root's choice and the child's choice don't have to be the same feature at all.

Formalize

At every node, a tree tries every feature's best threshold and keeps whichever gives the highest information gain. Feature importance just adds up that gain, weighted by how many samples reached each node, per feature, across the whole tree:

\text{importance}(f) = \sum_{\text{nodes splitting on } f} \frac{n_{\text{node}}}{n_{\text{total}}} \cdot \text{gain}_{\text{node}}

$\text{importance}(f)$ — the total importance score credited to feature $f$ across the whole tree.
$f$ — the feature being scored, i.e. whichever feature a node happened to split on.
$n_{\text{node}}$ — the number of samples that reached a given splitting node.
$n_{\text{total}}$ — the total number of samples in the whole tree.
$\text{gain}_{\text{node}}$ — the reduction in impurity that node's split achieved.

No separate explanation step needed
No perturbation, no gradients, no extra pass over the data — nothing this method needs isn't already produced by ordinary training.
The number exists the moment training ends
The importance score was already computed the moment the tree finished training; reading it back out is free.

Play

x1 importance

0.575

x2 importance

0.425

x1: used at the root (8/8 samples). x2: used only at the left child (4/8 samples), where it catches the one point x1 got wrong

x1 wins the root split, using every sample the tree has. x2 only ever gets used once, deeper in the tree, on a small subset — and still ends up with real, substantial importance, because that one split it does make is a clean, high-gain one.

Worked example

Eight points, two features. x1 orders the classes almost perfectly; one point is mislabeled relative to x1 alone:

The root split: x1, using all 8 samples
Threshold $4.5$ on x1 gives information gain $\approx0.549$ , weighted by all $8$ samples: $\tfrac{8}{8}\times0.549\approx0.549$ .
The left child: x2 catches the one exception
Among the 4 points that reached the left child, one has a label that x1 alone would get wrong. A split on x2 at threshold $3.5$ isolates it perfectly — gain $\approx0.811$ , weighted by $4$ of the $8$ total samples: $\tfrac{4}{8}\times0.811\approx0.406$ .
The right child needed nothing
All 4 points on the other side of the root split already share the same label — the best possible split there has gain $0$ , and contributes nothing to either feature's importance.

Normalizing $0.549$ and $0.406$ to sum to $1$ : x1 $\approx0.575$ , x2 $\approx0.425$ — x1 clearly the bigger contributor, but x2 far from irrelevant.

Checkpoint

Find the one point, among these three, whose label doesn't match what x1 alone would predict.

Pick a point to try it

Summary

\text{importance}(f) = \sum_{\text{splits on } f} \frac{n_{\text{node}}}{n_{\text{total}}} \cdot \text{gain}_{\text{node}}

Tree-based importance is the cheapest explanation method in this course — it requires training the model once, and nothing more. That's also its limit: it can only ever describe what the training procedure found useful for reducing impurity, which isn't automatically the same as what matters for a specific prediction, or what a human would consider the "true" cause. The next chapter asks for something tree importance can't give at all: a simple, human-readable rule that guarantees a specific prediction, not just a ranked list of contributing features.