A network is a chain

This whole curve is $y(x) = \sigma(w_2\, h(x) + b_2)$, where $h(x) = \sigma(w_1 x + b_1)$ is itself the output of a first neuron. Drag along it — the tangent line you see is the composed function's real slope, and it's built from nothing but two ordinary sigmoid derivatives.

Exactly Part I's chain rule, applied once per layer:

• $x$ — the network's input.
• $h$ — the first neuron's output, which feeds into the second neuron as its input.
• $y$ — the second neuron's output, the whole composed function's final value.
• $\frac{dh}{dx}$ — the first neuron's local slope: its own sigmoid derivative times $w_1$.
• $\frac{dy}{dh}$ — the second neuron's local slope with respect to its input $h$: its own sigmoid derivative times $w_2$.

1. Each neuron contributes its own local slope

   • $\frac{dh}{dx}$ is the first neuron's local slope — its own sigmoid derivative times $w_1$.
   • $\frac{dy}{dh}$ is the second neuron's local slope with respect to its input, which happens to be $h$ — its sigmoid derivative times $w_2$.
2. Multiply straight through the chain

   Multiply the two and you have the slope of the whole chain, with respect to the very first input.

The readout splits the product into its two factors. Neither factor alone is the answer — you need to multiply straight through the chain, exactly the way Part I's chain rule always worked, just with the first function's output value substituted into the second function's slope.

At $x=0.5$:

1. Compute the first neuron's output and slope

   $w_1 x + b_1 = 2(0.5)-1 = 0$, so $h = \sigma(0) = 0.5$ and $\frac{dh}{dx} = \sigma'(0)\cdot w_1 = 0.25 \cdot 2 = 0.5$.

2. Compute the second neuron's slope, at h

   $w_2 h + b_2 = -3(0.5)+1 = -0.5$, so $\frac{dy}{dh} = \sigma'(-0.5)\cdot w_2 \approx 0.235 \cdot (-3) = -0.705$.

3. Multiply the two slopes together

   $$\frac{dy}{dx} = -0.705 \times 0.5 = -0.353$$

   Matching the tangent line exactly at that point, and matching a plain numerical derivative to five decimal places.