Matrix multiplication as projection & subspace mapping

Here $A$'s column space has collapsed to a single line (drawn through $a$). Drag $b$ anywhere on the plane and watch $p$ — the closest point on that line to $b$ — barely budge in comparison. $p$ is the best $Ax$ can ever do.

The projection of $b$ onto the line spanned by $a$ is the point $p=ta$ that's closest to $b$, found by requiring the leftover piece to be perpendicular to $a$:

• $a$ — the vector spanning the column space (here, a single column, so the column space is a line).
• $b$ — the target vector being projected.
• $p$ — the projection: the closest point on the line to $b$.

1. Where this scalar comes from

   Writing $p=ta$ and demanding $a\cdot(b-p)=0$ (the residual is perpendicular to the line) gives $a\cdot b - t(a\cdot a)=0$, which solves directly to $t=(a\cdot b)/(a\cdot a)$.

2. This is exactly what Ax=b means geometrically

   When $A$ has one column $a$, solving $Ax=b$ means finding a scalar $x$ with $xa=b$ — possible only when $b$ already lies on the line. $p$ is the nearest point on that line when it doesn't.

3. The residual is never part of the column space

   $b - p$ is perpendicular to $a$ by construction, so it has zero component along the only direction $A$ can reach — it's the part of $b$ that gets permanently left behind.

Ax = b exactly solvable: no

Toggle between a $b$ that sits off the line (no exact solution — $p$ is a genuine approximation) and a $b$ that sits exactly on it (an exact solution exists, and $p=b$).

For $a=(2,1)$ and $b=(3,4)$:

1. Compute the scalar

   $a\cdot b = 2(3)+1(4)=10$, and $a\cdot a = 4+1=5$, so $t = 10/5 = 2$.

2. Compute the projection

   $p = 2(2,1) = (4,2)$ — the closest point on the line through $a$ to $b$.

3. Verify the residual is perpendicular to a

   $b-p = (3,4)-(4,2) = (-1,2)$. Check: $a\cdot(b-p) = 2(-1)+1(2) = -2+2 = 0$ — exactly perpendicular, confirming $p$ really is the closest point.