Maximum Likelihood Estimation (MLE)

Drag the candidate $p$ across the curve. Every point on it is the log-likelihood of these exact five flips under that one value of $p$ — and the curve has a single, unambiguous high point.

For $n$ independent Bernoulli trials with $k$ heads, the likelihood of the data under parameter $p$ is

• $L(p)$ — the likelihood: how probable the observed data is, as a function of $p$.
• $k$ — the number of heads actually observed.
• $n$ — the total number of flips.

Products of small numbers underflow fast, so in practice everyone maximizes the log-likelihood instead — same maximizer, better numerics:

1. The log is monotonic, so the peak doesn't move

   $\log$ is strictly increasing, so whatever $p$ maximizes $L(p)$ also maximizes $\ell(p)$ — taking the log never changes where the maximum sits.

2. Setting the derivative to zero gives a closed form

   $$\frac{d\ell}{dp} = \frac{k}{p} - \frac{n-k}{1-p} = 0$$

   1. Move the second term across and cross-multiply: $\frac{k}{p} = \frac{n-k}{1-p} \implies k(1-p) = p(n-k)$
   2. Expand both sides: $k - kp = pn - pk$
   3. The $-kp$ on the left and $-pk$ on the right are the same term, so they cancel, leaving $k = pn$

   Dividing both sides by $n$ gives

   $$\hat{p} = \frac{k}{n}$$

   — the maximum likelihood estimate is exactly the observed proportion of heads, no search required.

3. This is the general MLE recipe

   Write the likelihood of the data under a parameter, take its log, and find where the derivative vanishes — the same three steps work for far more complex models than a coin.

The curve's exact peak sits at $p=0.8$, matching $\hat{p}=k/n$ exactly — drag away in either direction and the log-likelihood only gets worse, confirming this closed form really is the unique maximizer.

Five flips: H, H, T, H, H — so $k=4$ heads out of $n=5$.

1. Apply the closed-form MLE

   $$\hat{p} = \frac{k}{n} = \frac{4}{5} = 0.8$$
2. Confirm it by evaluating the log-likelihood at the estimate

   $$\ell(0.8) = 4\log(0.8) + 1\log(0.2) \approx 4(-0.2231) + (-1.6094) \approx -2.502$$
3. Check a nearby value scores worse

   $$\ell(0.6) = 4\log(0.6) + 1\log(0.4) \approx 4(-0.5108) + (-0.9163) \approx -2.960$$

   Worse (more negative) than $-2.502$ — exactly what "$0.8$ is the maximizer" predicts.