The forward pass

Drag either input. Every neuron computes its own weighted sum from the exact same $(x_1, x_2)$ — they just disagree about which weights and bias to use. Watch the ring around each one: it lights up the moment that neuron's output is positive.

A layer is just several neurons run in parallel on the same input:

• $z_i$ — neuron $i$'s raw weighted sum, before activation.
• $w_i$ — neuron $i$'s own weight vector.
• $b_i$ — neuron $i$'s own bias.
• $x$ — the shared input vector fed to every neuron in the layer.
• $a_i$ — neuron $i$'s activation, the layer's output for that neuron.

1. Nothing new inside each neuron

   $z_i$ is exactly the perceptron's weighted sum, computed once per neuron.

2. A layer is several of these in parallel

   A layer of 3 neurons on 2 inputs is 3 independent weighted sums, each fed through the same activation.

3. Stacking layers builds a network

   Feed one layer's outputs into the next layer's inputs, and stack enough of these, and that's a neural network.

Push $x_1$ negative and $x_2$ positive. Some neurons' sums go negative and their ReLU (short for Rectified Linear Unit) output flatlines at exactly 0 — dead for this input, no matter how negative their sum gets. Others stay positive and keep responding linearly. Same inputs, three completely different reactions.

At $(x_1, x_2) = (2, 2)$:

1. Neuron A: w = (1, -1), b = 0

   $z = 2 - 2 = 0$, so $\text{ReLU}(0) = 0$.

2. Neuron B: w = (0.5, 0.5), b = -1

   $z = 1+1-1 = 1$, so its output is $1$.

3. Neuron C: w = (-1, 2), b = 0.5

   $z = -2+4+0.5 = 2.5$, output $2.5$.

4. Compare the three

   Same $(x_1,x_2)$, fed through three different weight vectors, three completely different answers — that's the entire forward pass.