Expectation, variance & covariance

Each row of the table is one outcome; each bar is that outcome's own contribution $p_i \cdot x_i$ (or $p_i \cdot y_i$) to the mean. Toggle between $X$ and $Y$ — the bars change, but they always add up to exactly the number printed underneath.

The expectation (mean) of a discrete random variable weights every outcome by how likely it is:

• $\mathbb{E}[X]$ — the expected value of $X$: its probability-weighted average.
• $p_i$ — the probability of outcome $i$.
• $x_i$ — the value $X$ takes on outcome $i$.

Variance measures spread around that mean, and covariance measures whether two variables move together:

• $\mathrm{Var}(X)$ — the expected squared deviation of $X$ from its own mean.
• $\mathrm{Cov}(X,Y)$ — the expected product of $X$'s and $Y$'s deviations from their own means.

1. Variance is a special case of covariance

   $\mathrm{Var}(X) = \mathrm{Cov}(X,X)$ — pairing a variable with itself just squares the deviation.

2. The sign of covariance is the whole story

   A positive $\mathrm{Cov}(X,Y)$ means $X$ and $Y$ tend to sit on the same side of their means together; negative means they tend to sit on opposite sides. The magnitude alone is hard to read without normalizing by spread (which is what correlation does).

3. Every outcome pulls in one direction or another

   An outcome where both $X$ and $Y$ are above their means, or both below, contributes positively to the covariance sum; an outcome where one is above and the other below contributes negatively.

Switch between $\mathrm{Var}(X)$, $\mathrm{Var}(Y)$, and $\mathrm{Cov}(X,Y)$: the bars are the exact terms being summed, and outcome 3 — where $x=3$ sits right at $\mathbb{E}[X]$ — contributes nothing to either variance or covariance, because its deviation is zero.

1. Find E[X] and E[Y]

   $$\mathbb{E}[X] = 0.1(1) + 0.3(2) + 0.2(3) + 0.3(4) + 0.1(5) = 3.0$$$$\mathbb{E}[Y] = 0.1(2) + 0.3(1) + 0.2(4) + 0.3(3) + 0.1(5) = 2.7$$
2. Find Var(X)

   $$\mathrm{Var}(X) = 0.1(1{-}3)^2 + 0.3(2{-}3)^2 + 0.2(0)^2 + 0.3(1)^2 + 0.1(2)^2 = 0.4+0.3+0+0.3+0.4 = 1.4$$
3. Find Cov(X, Y)

   The five outcomes are $(x_i,y_i)$: $(1,2)$, $(2,1)$, $(3,4)$, $(4,3)$, $(5,5)$, with the same probabilities $0.1, 0.3, 0.2, 0.3, 0.1$ as above. Subtract each mean and multiply the deviations:

   • $(1{-}3)(2{-}2.7)=(-2)(-0.7)=1.4$
   • $(2{-}3)(1{-}2.7)=(-1)(-1.7)=1.7$
   • $(3{-}3)(4{-}2.7)=(0)(1.3)=0$
   • $(4{-}3)(3{-}2.7)=(1)(0.3)=0.3$
   • $(5{-}3)(5{-}2.7)=(2)(2.3)=4.6$

   Weighting each by its outcome's probability and summing:

   $$\mathrm{Cov}(X,Y) = 0.1(1.4) + 0.3(1.7) + 0.2(0) + 0.3(0.3) + 0.1(4.6) = 1.2$$

   Positive — when $X$ swings away from its mean, $Y$ tends to swing the same direction.