Sim-to-real transfer & physics engines (MuJoCo / Isaac)

Same gain, same target, same number of control steps. In sim the arrow lands on the target dot. In "real," with the actuator delivering less motion per command than assumed, the same gain undershoots — not because anything is broken, just because the model the controller was tuned against wasn't quite right.

A proportional controller commands motion equal to gain times error, but the actuator only delivers a fraction of what's commanded. That turns the position error into a geometric sequence:

• $e_T$ — the remaining distance to the target after $T$ control steps.
• $e_0$ — the starting distance to the target.
• $f$ — the friction factor: the fraction of commanded motion the actuator actually delivers ($f=1$ is the sim's assumption).
• $K$ — the controller's proportional gain.
• $T$ — the fixed number of control steps before checking success.

1. Sim assumes f = 1

   A gain tuned so that $|1-K|$ is small enough to converge within $T$ steps in sim is really only tuned for $f=1$ — it has no reason to also work for a smaller $f$.

2. Domain randomization means tuning against a range of f, not one value

   Training against $f=1$ produces a controller that's brittle to exactly one number. Training against many sampled $f$ values (0.6, 0.8, 1.0, ...) produces a gain that keeps $e_T$ small across the whole range — the real robot's actual $f$ just needs to be somewhere in that trained range.

Identical gain, identical target, identical step budget — only the friction changed, and that alone flips the outcome from success to failure.

1. Sim: f=1, K=0.5, four steps

   $e_4 = 4 \cdot |1 - 1(0.5)|^4 = 4 \cdot 0.5^4 = 4(0.0625) = 0.25$ — comfortably inside the 0.5 tolerance.

2. Real: same gain, f=0.6

   $e_4 = 4 \cdot |1 - 0.6(0.5)|^4 = 4 \cdot 0.7^4 = 4(0.2401) = 0.9604$ — the exact same controller, the exact same math, now fails the tolerance.

3. Compensating for the known friction closes the gap

   Solving $4|1-0.6K|^4 \le 0.5$ for $K$: divide both sides by 4 to get $|1-0.6K|^4 \le 0.125$; take the fourth root, $|1-0.6K| \le 0.125^{1/4} \approx 0.595$; so $1-0.6K \le 0.595$, giving $K \ge (1-0.595)/0.6 \approx 0.676$. Any gain past roughly $K \approx 0.68$ brings the real robot back under tolerance in the same four steps — the controller doesn't need a better simulator, it needs a gain that assumes less friction than sim did.