Grad-CAM

This is the same minimal CNN (short for Convolutional Neural Network)'s 4×4 feature map after ReLU (short for Rectified Linear Unit) — the exact tensor that gets pooled and fed to the dense layer. Grad-CAM reweights this map by how much each of its cells actually mattered to the predicted class, then clips anything left negative to zero.

Grad-CAM has two steps. First, weight each feature-map channel by how much it mattered to class $c$'s logit — averaging that channel's gradient over every spatial location:

• $\alpha_k^c$ — the weight for channel $k$: how much that channel matters to class $c$'s logit.
• $H, W$ — the feature map's height and width, i.e. its number of spatial locations.
• $y^c$ — class $c$'s logit, the model's raw output score for that class.
• $A^k_{ij}$ — the feature map's activation for channel $k$ at spatial location $(i,j)$.
• $k$ — the index over feature-map channels.
• $c$ — the class being explained.

Then combine the channels using those weights, and clip to only the positive evidence:

• $\text{Grad-CAM}^c$ — the final heatmap for class $c$: a weighted, positive-only combination of the feature-map channels.

1. With one filter, one channel weight

   With only one convolution filter, there's just one channel $k$, so $\alpha^c$ collapses to a single number.

2. But that number still depends on the class

   That single number still depends on which class $c$ you ask about, which is exactly what makes Grad-CAM class-discriminative instead of a fixed, class-blind map.

Switch classes. The weight flips from $+0.5$ to $-0.5$ — same feature map, opposite sign — because "Edge detected" and "No edge" read the identical evidence with opposite-signed dense weights. Watch what that does to the CAM itself once ReLU is applied.

The feature map is $\begin{pmatrix}0&0&3&3\end{pmatrix}$ in every one of its 4 rows — the same tied activations from the saliency chapter, one level before pooling:

1. The channel weight, for each class

   Every 2×2 pooling window here is a 4-way tie (all $0$s or all $3$s), so nudging any one cell up by $\epsilon$ makes it that window's new unique max — the pooled output rises by $\epsilon$ — while nudging it down by $\epsilon$ leaves the other three tied cells still winning, so the pooled output doesn't move at all. That one-sided response is what the numerical gradient $\frac{\text{plus}-\text{minus}}{2\epsilon}$ picks up:

   • For "Edge detected" (dense weight $+1$ on every flattened position), one cell's plus-side logit rises by $1\times\epsilon$ while its minus-side logit is unchanged, giving gradient $\frac{\epsilon-0}{2\epsilon}=0.5$ for every cell (the same tie-breaking happens in every window). Averaging 16 identical $0.5$s over all 16 cells just gives back $0.5$, so $\alpha^{\text{edge}} = 0.5$.
   • For "No edge"'s $-1$ dense weight, the same argument gives $-0.5$ everywhere instead, so $\alpha^{\text{no edge}} = -0.5$.
2. Grad-CAM for 'Edge detected'

   $\text{ReLU}(0.5 \times \begin{pmatrix}0&0&3&3\end{pmatrix}) = \begin{pmatrix}0&0&1.5&1.5\end{pmatrix}$ — the two right-hand columns, exactly where the edge evidence lives.

3. Grad-CAM for 'No edge'

   $\text{ReLU}(-0.5 \times \begin{pmatrix}0&0&3&3\end{pmatrix}) = \text{ReLU}\begin{pmatrix}0&0&-1.5&-1.5\end{pmatrix} = \begin{pmatrix}0&0&0&0\end{pmatrix}$ — completely zero. There's no region of this image that positively supports "no edge," so ReLU throws all of it away.