The rudder
and the world model

1 Three versions of the same question

The question is a single one: does the model know where it is going before it goes? Each version answers in its own way, and each was measured against the previous one under the same protocol (chapter 06): same frozen data, 700 steps, seed 1337, best checkpoint, control = cosine 1.0 with validation 5.2706.

2 What v1 and v2 measured

The result that organizes the chapter: predicting the next state in the latent helps the generator (−0.90%, one seed), and predicting the token's embedding does not help (+1.41%). The difference between the two is the target: the hidden state carries context; the row of the embedding table does not. It is DINO's self-distillation with stop-gradient in place of the EMA teacher — and the run with the real teacher has answered: the EMA buys nothing — it costs 3 points against stop-gradient, on one seed (chapter 04).

The additive term at generation: a clean negative result

The first idea for using the rudder was to add to the logits a term α · cos(E[v], Pred(h)). The test (evaluate_compass.py --only-alpha) measured the validation loss with three directions in place of Pred: the predictor's, the identity (h itself) and a random direction. The predictor was worse than the identity, and the identity worse than random. Adding a direction to the logits makes things worse, whatever the direction — because it distorts a calibrated distribution. That is what condemned additive inference and motivated the energy (chapter 09): the predicted direction serves to choose among candidates, not to reweight all of them.

3 The world model, piece by piece

The table the queue will fill

The first run, read

wm1-sz1 closed with both sides of the coin. The world model works: error 0.150 against 0.948 for copying the state, cosine 0.93 against 0.58 — and it understands sequence, as the violation of expectation below shows. But validation went to 5.7761, +9.59% over the control: the worst quality cost of all the runs. The hypothesis, written before the control closes: with weight 1.0 and end-to-end training, the encoder shapes the states to be predictable from the action — a shortcut — and loses what the cross-entropy needs. wm1det (target with stop-gradient, without SigReg-z) separates the two things; at step 99 it was already 0.63 nats below wm1-sz1 at the same step. By the house rule, the answer is not to discard: two runs with weight 0.1 (wm0.1-sz1 and wm0.1n2-sz1-lat1o, the combination with the deep rudder) entered the queue.

It is the ball that vanishes from the video, in text: the same sentence backwards makes the map's error rise 32% while the identity barely notices (−3%). The world model learned something about the dynamics of text that copying the state does not capture — the first "common sense" measured in this project, and it came in the run that cost the generator the most.

The latent variable, read

wm1n2z16-sz1 had the lowest cost of the end-to-end runs (+7.00%) and the largest map gain in the table — and the gain is false. ζ is read from the target through a 16-dimensional channel, regularized by SigReg to be N(0, I); the bet was that a narrow, Gaussian channel would only carry "which future", not "the whole future". Measured on validation, with the same states:

With the ζ that inference has, the map is the floor. The channel carried the target — and SigReg on the ζ did not close the channel: the mean stayed at 0.00 but the standard deviation at 1.44 and the norm at 5.7 (N(0, I) would give 1.0 and 4.0). The symptom shows up in everything that uses ζ = 0: the violation of expectation vanishes (+3 / −4 / −1 beyond the identity, against +25 / +41 / +39 for wm1n2-sz1) and level 2 becomes worse than copying the block. It is the talk's collapse through another door: not the constant representation, but the trivial prediction via a shortcut. The fix is in the queue: ζ-dropout (--world-latent-dropout 0.5) — half the batches with ζ sampled, so that the predictor has to work without the target, and ζ can only carry what a sample would also carry. Meanwhile, this model's rudder governs anyway (chapter 12: 0.383 against 0.265 for free), because the planner samples ζ and the cost on the real state does not depend on the map.

ζ-dropout closed the channel. With ζ sampled, as at inference, the map errs 0.222 against 1.110 for the floor — before, it erred 1.260 against 1.671, that is, nothing. The difference between ζ inferred and ζ sampled fell from 33× to 1.5×: what ζ still carries is the choice among futures, not the future. The cost to the generator stayed the same (+7.60% against +7.00%). And ζ came closer to N(0, I) (standard deviation 0.70, norm 2.75) — SigReg on the ζ starts to have an effect once the predictor can no longer depend on them. The violation of expectation came back with it: with ζ = 0, +22.6 / +31.4 / +30.4 beyond the identity (against +3 / −4 / −1 in the leaked channel) — the map understands sequence again. And chapter 12 closes the cycle: on this map the rudder governs on both axes (0.324 against 0.243, PPL 51.7 against 62.6).

The two stages, read

The talk's recipe — "use DINO as the encoder, then train a world model and plan" — is wm-sobre-dino: the encoder of run reg1-dino1-lat1o frozen, only the world model's heads learning, 532 thousand parameters, eleven minutes. The map comes out worse than the end-to-end one (MSE 0.665 against 0.15; cosine 0.73 against 0.93) and better than the floor by a wide margin (+0.80). What it buys is what no end-to-end run bought: zero cost to the generator, because the generator is not touched. It is the opposite pair of wm1-sz1: there the map is excellent and the generator pays 9.59%; here the map is good and the generator pays nothing. Between the two, wm1det (+1.11%, cosine 0.79). The choice between them is the choice between a more precise map and an intact generator — and chapter 12 says that, to govern, the map does not need to be the most precise: on this one, the rudder gives 0.436 against 0.281 for free, the largest effect of all the maps. The violation of expectation is there too (+10.1 / +31.6 / +32.7 beyond the identity): the map trained in eleven minutes on top of an encoder that never saw a world model understands sequence.

The control, read

wm1det — the same world model with the target without gradient and without SigReg on the states, the "old shortcut" the LeWorldModel recipe dispenses with — closed at 5.3291, +1.11%, with a map gain of +0.87 over the floor (MSE 0.535 against 1.406; cosine 0.79 against 0.51). Against wm1-sz1: eight percentage points less in cost, and a map that predicts almost as well. Criterion (2) of the box below fired: end-to-end training with SigReg-z, with weight 1.0, does not pay for what it costs in this model. What it buys is a more predictable space (cosine 0.93 against 0.79) — and chapter 12 shows that this space governs when the cost has scale; what it charges is the generator. Two honest readings fit at the same time: the stop-gradient did not collapse (the identity floor remains far from the map's error, and the SigReg of the states, measured without training, stayed at 10× the Gaussian floor against 12.8× for wm1-sz1), and the end-to-end recipe may be right with the wrong weight. wm0.1-sz1 has said: with weight 0.1 the cost falls from +9.59% to +3.25%, and the map keeps beating the floor (+1.17, cosine 0.71 against 0.33). The weight was most of the bill; the two points left between 0.1 end to end and the stop-grad are the price of SigReg on the states, which has not paid for itself in any run yet. The violation of expectation on wm1det gives the same picture, a little weaker: another document +10.7%, shuffled +26.0%, reversed +27.7% beyond the identity (against +17.6 / +36.3 / +35.3 for wm1-sz1, +25.0 / +41.3 / +39.3 for wm1n2-sz1 and +16.0 / +41.4 / +39.4 for wm1n2e-sz1). With weight 0.1 (wm0.1-sz1): +9.4 / +20.1 / +22.8 — the lighter map understands less sequence, in the same order. All the level-1 maps understand sequence; only the one with the leaked ζ, with ζ sampled, does not.

4 Inference: how the rudder becomes a rudder

With the world model trained, generation changes in nature: instead of adding a bias (the negative result of §2), the system imagines each candidate and chooses by energy. The modes, all selectable in the «Antivision · rudder» panel (chapters 09 and 10):

And on top of the modes, three flags from the reading of LeCun: the task cost and the guardrails can be learned heads from three sentences (costs.py); level 2 can plan its own abstract action (plan2_steps); and the planner can stay quiet when the model has no doubt (plan_if_entropy). The comparison of all this on a trained model is chapter 12.

The cross-test table

The evaluate_compass.py test: three headings × four prompts. For each generation, the affinity with each heading (neighborhood in the 400 nearest tokens) and the perplexity of the generated text. A rudder works if the diagonal of the affinity matrix dominates and the perplexity does not explode.

Effect (diagonal − off-diagonal): +0.017 with β = 8; 0.000 with β = 16. The rudder, with the cosine-to-heading cost, does not govern this model: the three rows are almost equal, and with β = 16 they are identical — the planner always chooses the same candidate, the most probable one, because the task energy does not distinguish the candidates. The map predicts well and the cost does not point. It is exactly the gap the learned cost head (chapter 12) was made to close; the modes table, with the independent judge, says whether it closes.

5 Correcting the map

When the plan does not give the predicted result, what gets corrected is the world model (chapter 09, §3.3). The smoke test, on a two-step world model, gave a surprise/floor ratio of 0.907 → 0.719 on a held-out set. The number that counts is the same test on a trained model, where the map is already good and the margin is small:

What the smoke test predicted, happened: with the map already at 0.16, the correction memorized its own trajectories (0.149 → 0.095 on the generated text) and did not generalize (0.161 → 0.159 on the test). It is the "critic overfitting" the talk says to watch for, and it appears exactly where the margin is small. The loop is right and so is the reading criterion; what this map lacks is systematic error to correct.

Chapter 06 tells how all these numbers were measured — and the eleven mistakes the method caught along the way.