Maximum A Posteriori (MAP) estimation

The faint curve is the plain likelihood from the flips alone; the bold curve adds in a tails-favoring prior. Dial the prior's strength up and watch the bold curve's peak slide away from the faint curve's — that gap is the prior's influence.

Bayes' rule turns a likelihood into a posterior by multiplying in a prior belief over the parameter itself. The MAP estimate maximizes that posterior instead of the raw likelihood:

• $\hat{p}_{\text{MAP}}$ — the maximum a posteriori estimate.
• $L(p)$ — the data likelihood, same as in plain MLE.
• $\pi(p)$ — the prior density over $p$, encoding belief held before seeing any data.

Using a Beta$(\alpha,\beta)$ prior — encoded as $\alpha-1$ imagined extra heads and $\beta-1$ imagined extra tails — gives a closed form that looks just like the MLE with pseudo-counts folded in:

• $\alpha,\beta$ — the prior's pseudo-counts: imagined extra heads and tails, respectively.
• $k, n$ — the actually observed heads and total flips, exactly as in MLE.

1. A uniform prior (α=β=1) reduces exactly to MLE

   Plug in $\alpha=\beta=1$ and the pseudo-counts vanish: $\hat{p}_{\text{MAP}} = k/n = \hat{p}_{\text{MLE}}$.

2. MAP is a compromise, not a replacement

   $\hat{p}_{\text{MAP}}$ always sits between the prior's own mean $\alpha/(\alpha+\beta)$ and the MLE — more data pulls it toward the MLE; a stronger prior pulls it toward the prior mean.

3. The normalizing constant never matters

   The true posterior divides by $P(\text{data})$, but that constant doesn't depend on $p$, so it can never change where the maximum sits — only the unnormalized numerator matters for MAP.

At strength 0 the two peaks coincide exactly — an uninformative prior changes nothing. Every notch up adds more imagined tails-observations, and the MAP estimate keeps sliding toward the prior's own mean.

Same four flips as the MLE chapter's toy — H, H, H, T, so $k=3$, $n=4$ — now with a Beta($\alpha{=}2,\beta{=}4$) prior.

1. The MLE, ignoring the prior entirely

   $$\hat{p}_{\text{MLE}} = \frac{3}{4} = 0.75$$
2. Apply the MAP closed form

   $$\hat{p}_{\text{MAP}} = \frac{k+\alpha-1}{n+\alpha+\beta-2} = \frac{3+2-1}{4+2+4-2} = \frac{4}{8} = 0.5$$
3. Compare to the prior's own mean

   The prior alone believes $\alpha/(\alpha+\beta) = 2/6 \approx 0.333$. The MAP estimate, $0.5$, lands exactly between that prior belief and the MLE's $0.75$ — pulled by the prior, not overridden by it.