Binary cross-entropy loss

Drag the point — it's your model's predicted probability that this true-positive example is positive. Near 1 the loss barely moves. Down near 0, it shoots upward without any sign of leveling off.

That's cross-entropy loss. For one example with true label $y$ and predicted probability $p$ —

• $L$ — the cross-entropy loss for one example.
• $y$ — the true label, exactly $0$ or $1$.
• $p$ — the model's predicted probability that the example is positive.

1. Only one term is ever active

   Since $y$ is exactly 0 or 1, exactly one of the two terms in the brackets is ever nonzero — the loss only ever looks at the probability assigned to the true label.

2. Confidently wrong predictions blow up

   A probability approaching 0 for the correct answer sends $\log p$ toward $-\infty$ — and the leading minus sign turns that into a loss with no ceiling at all.

Watch the comparison: squared error tops out at $1$ no matter how wrong the prediction gets, but cross-entropy keeps climbing. A model that's confidently wrong gets punished far harder than one that's just wrong.

True label $y=1$, predicted $p=0.1$ — confidently wrong:

1. Compute the cross-entropy loss

   $$L = -\log(0.1) \approx 2.303$$
2. Compare it to squared error

   Squared error for the same case is only $(1-0.1)^2 = 0.81$ — cross-entropy is already nearly three times larger, and would keep growing the more confident the wrong prediction became.