Logistic regression & the sigmoid

Hook

Linear regression outputs any number at all. What if the answer you actually want is a probability — something that has to stay between 0 and 1?

Intuition

Drag the point along the curve. Near the middle it swings quickly from "probably not" to "probably so" — but far to either side, it barely moves at all, no matter how much further you drag.

Formalize

That S-shaped squashing function is the sigmoid —

\sigma(z) = \frac{1}{1 + e^{-z}}

$\sigma(z)$ — the sigmoid function: squashes any real number into a probability between 0 and 1.
$z$ — the raw input score being squashed, before it's turned into a probability.

Applied to the same linear score
Feed in the same linear score from Chapter 1 of this part, $z = wx + b$ . Together, $\sigma(wx+b)$ turns any real number into a valid probability.
Where it crosses 0.5 is the decision boundary
The point where $\sigma(wx+b)$ crosses exactly $0.5$ — where $wx+b=0$ — is the decision boundary.

Play

z = 1.00(2.0) + (-5.00) = -3.00 → P = 0.05

w1.00b-5.00

Drag $w$ and $b$ and watch the boundary itself move: $b$ shifts it sideways, $w$ controls how sharply the curve turns from 0 to 1 around it.

Worked example

With $w=1, b=-5$ :

Evaluate at x = 5
$z = 1(5) + (-5) = 0$ , so $\sigma(0) = 0.5$ exactly — the boundary.
Evaluate at x = 8
$z = 1(8) + (-5) = 3$ , so $\sigma(3) \approx 0.953$ — confidently on the "yes" side.

Checkpoint

Balance w and b until the decision boundary — where P = 0.5 — lands exactly at x = 5.

P(x = 5) = 0.95

w1.00b-2.00

Move a slider to try it

Summary

\sigma(z) = \frac{1}{1+e^{-z}}, \qquad z = wx+b

Logistic regression is linear regression's score, squashed into a probability. Its derivative is exactly the chain rule from Part I — $\sigma$ applied to a line, differentiated as $\sigma'(z)\cdot w$ . Next: the loss function built specifically for this kind of output.