Recommender systems & matrix factorization

Toggle between one latent factor and two. With one, the reconstructed grid never quite matches the real ratings — it's making the same kind of mistake in the same place, no matter how long it trains. With two, it locks onto the real pattern.

Give every user $i$ and every item $j$ a small vector of latent factors, and predict a rating as their dot product:

• $\hat{r}_{ij}$ — the predicted rating of item $j$ by user $i$.
• $u_i$ — user $i$'s latent factor vector.
• $v_j$ — item $j$'s latent factor vector.
• $r_{ij}$ — the actual observed rating, when one exists.
• $\mathcal{L}$ — the total squared-error loss, summed only over ratings that are actually observed.

1. Train only on ratings that exist

   Gradient descent updates every $u_i$ and $v_j$ to reduce that squared error — but only summing over ratings that actually exist. The missing entry never appears in the loss at all.

2. Predicting a missing entry is just the same dot product

   Once training finishes, predicting a rating that was never observed is nothing special — just plugging its user and item vectors into the same $u_i \cdot v_j$ used everywhere else.

With one factor, $U_3$'s rating of $I_3$ — a genuine $5$ — comes out well under $2$, and no amount of extra training time fixes it. With two factors, the same entry reconstructs almost exactly. One number wasn't enough to describe two people who like opposite things.

Two real taste groups: $U_1$ and $U_2$ like $I_1$/$I_2$ and dislike $I_3$; $U_3$ is the mirror image:

1. One factor plateaus at a real error, not a training failure

   A single number per user and per item can only express "more of one thing," not two independent directions of taste. Training converges — the loss stops moving — but it converges to a genuinely wrong answer, because a rank-1 model cannot represent this data no matter how long it runs.

2. Two factors reconstruct every observed rating almost exactly

   With one extra dimension — and a starting point that isn't identical across dimensions, so the second one doesn't just copy the first — the total squared error over all 8 known ratings drops to essentially $0$.

3. The missing rating gets a specific, sensible prediction

   $U_2$'s hidden rating of $I_2$ comes out to about $3.21$ — between $U_1$'s $4$ and something lower, exactly reflecting that $U_2$ likes $I_1$ slightly less than $U_1$ does.