The whole project, piece by piece, aligned with the four patterns of the world model: joint-embedding, energy, regularization and predictive control — and what is still hybrid, stated with numbers.
The slide "Recommendations to my fellow AI scientists" has four lines, each an abandon X in favor of Y. This chapter is an audit: for each line, which piece of our code implements it, which flag turns it on, and whether it has been measured yet. Where the project still does X and Y at the same time, that is written down — because the house rule is keep and improve, without regression, not swapping what works for a promise.
| The slide says | Our piece | Flag | Status |
|---|---|---|---|
| Abandon generative → joint-embedding |
The world model predicts in latent space: Pred( |
--world --sigreg-z |
Hybrid. wm* runs in the queue. |
| Abandon probabilistic → energy-based |
At generation, candidates are reordered by an energy: guardrail (−log p, accumulated over the imagined future) + task (1 − cos to the goal). The logit stops being the decision and becomes one of the terms. | compass.mode = energy | gradiente | mpc |
Implemented. Measured only on the control without WM (chapter 06, §4: the additive term hurts). |
| Abandon contrastive → regularized |
There was never a negative pair here. Anti-collapse is entirely by regularization: cosine on the embedding table, SigReg, VICReg, Barlow. | --reg-cos --sigreg --vicreg --sigreg-z |
Measured (chapter 03 and the table in §5). |
| Abandon RL → predictive control (MPC) "RL only when planning doesn't yield the predicted outcome, to adjust the world model" |
Beam search in the latent is MPC in miniature. And the second half of the sentence became a script: it measures the surprise (predicted × real), and corrects only the world model, with the encoder frozen. The generator is not touched. | _planejar(), correct_world.py, /nosso/trajetoria |
Implemented; smoke test in §4. |
The talk argues that predicting symbols is not enough to build a world model. Even so, our token decoder does three things the world model alone does not: it produces text (the latent is not readable), it provides the guardrail −log p that keeps the planner from leaving the distribution, and it is the only ruler comparable with everything that exists (bits/character). Removing the token head today would be removing the measuring instrument. And there is a stronger reason, which the interview gives (chapter 10, §3): the intractability that motivates JEPA holds for continuous signals; in text, discrete and finite, the softmax over the next token is the distribution over the futures. The plan is to invert its role — from decision to energy term — and that is what the energy and mpc modes already do.
The "Outlook" slide draws a hierarchical JEPA: encoders (half-circles), predictors conditioned on an action (green arrows with the red
abstrair() at level 2 — the block mean, optionally corrected by a learned encoder (--world-enc2). Green = action-conditioned predictors: at level 1 the action is the embedding of the candidate token; at level 2 it is the intent, the mean of the embeddings of the goal you wrote. Red = where the predicted meets the real: in training it is the MSE against the identity floor; at generation it is the energy that orders the candidates, and the surprise that gets recorded. The dashed arrow is hierarchical planning: level 2 imagines the next block, and that point becomes level 1's short-range target — instead of the final goal, too far away for a three-token horizon to say anything.A detail that cost a potential bug: the planner's subgoal has to go through the same level-2 encoder that training used. If training learned to predict over abstrair(z) and the planner feeds mundo2 the raw mean, it is planning in a space it has never seen. That is why abstrair() is a method of the model, called from both sides, and not a line duplicated in each script.
Up to here the abstract state was the block mean. The mean works — "higher levels predict further with less detail" comes for free, because the mean discards detail. But it discards always the same detail. The slide draws a dedicated encoder at level 2, and our version is the smallest one that makes sense: a residual correction mundo2 trains
# hierarchy with the learned level-2 encoder (run reg1-wm1n2e-sz1) .venv\Scripts\python.exe model\train.py --size mini --modern --world 1 --world-levels 2 --world-enc2 --sigreg-z 1 --data duelo_200mb --run duelo
A learned encoder over a target that is also learned is the recipe for collapse: mundo2 "gets it right" every time. What holds it back is the SigReg on the level-1 states, which feed into abstrair — and the identity floor in the report: if ganho_mundo2 is large and the raw mean's is too, the encoder did nothing; if the gain comes with piso_mundo2 plummeting, it is collapse disguised as accuracy.
It happened. In the wm1n2e-sz1 run the block floor fell from 0.160 to 0.013 and the level-2 gain from +0.072 to +0.006 — the same fraction, on a scale thirteen times smaller (chapter 11). The first fix tried was the one the recipe prescribes for level 1: SigReg on the abstract states too (--sigreg-s2). It recovered part of the scale (floor 0.013 → 0.068) at the price of almost orthogonal consecutive blocks and +11.32% on the generator — the level-1 recipe does not serve level 2, which needs continuity between blocks. The second fix, scale-only (--world-enc2-norm), collapsed through the other door: with the norm fixed, every block went to the same direction (cosine 1.000). Three attempts, three shortcuts; the raw mean is the honest abstraction on this budget, and the right anti-collapse for level 2 remains open (chapter 11).
From the talk: "use DINO as an encoder, then train a world model and do planning". It is not what LeWorldModel does (there the encoder and the predictor learn together), and both are right for different questions. The two-stage one asks: is the space of a self-distilled encoder better for prediction than a space trained together with the predictor? To answer, we need to train the world model on top of a frozen encoder — and our DINO encoder is the run reg1-lat1o-dino1.
# world model on top of the DINO encoder, frozen (run wm-sobre-dino) .venv\Scripts\python.exe model\train.py --size mini --modern --world 1 --world-levels 2 --world-enc2 --sigreg-z 1 ^ --base mini-moderno-reg1-lat1o-dino1-duelo --freeze-encoder --data duelo_200mb --run duelo
With the encoder frozen, only 532 thousand trainable parameters remain out of 7.9 million (the 7.4 M of mini plus the world model heads), and there is no backpropagation through the Transformer — the run costs a fraction. That run's validation loss is, by construction, the base's: the generator did not change. What one reads is the wm ganho column of the report, against the same column of the end-to-end run.
The last line of the slide has two halves, and it is the second one that matters here: "use RL only when planning doesn't yield the predicted outcome, to adjust the world model or the critic". Translated into our system:
The bench records the surprise token by token in every generation from a -wm model, and shows the mean, the floor and the peaks in the bench's «Antivision · rudder» panel. The script closes the loop:
# measures surprise on real text and on what the model generated; --finetune corrects only the WM .venv\Scripts\python.exe model\correct_world.py --model mini-moderno-reg1-wm1n2-sz1-duelo --heading "futebol gol" --finetune
It saves the result as <modelo>-corrigido, one more training run on the bench. The original is not touched. And it measures before and after on a separate set — because the obvious trap is the map memorizing its own trajectories, which is our version of the "critic overfitting" the talk says to watch for.
Before the wm* run finished, the loop was tested on a world model with two training steps — almost random, on top of the frozen encoder of reg1-lat1o. It is not a scientific result; it is proof that the mechanism does what it says. The ratio is surprise ÷ identity floor: below 1, the map predicts better than "copying the state".
| Source | Ratio before | Ratio after | Reading |
|---|---|---|---|
| text generated by the model (seen in the correction) | 1.303 | 0.713 | the map was worse than the identity; it no longer is |
| separate test | 0.907 | 0.719 | generalized: dropped by almost the same amount |
Two warnings. First: a two-step WM had everything to gain — the drop on a properly trained model will be smaller, and may be zero. Second: the surprise peaks in this test all fell on digit sequences, because the tokenizer splits digits (chapter 01) and an almost-random map knows nothing about numbers. On a trained model, the peaks are the interesting information: they say where the map is wrong.
| Run | vs control | What it says about the alignment |
|---|---|---|
cosine + deep compass (reg1-lat1o) | −0.90% | predicting the next state in the latent helps the generator. First run to beat the control. One seed: caution. |
SigReg on the embedding table (sig1) | +1.51% | effective rank 245.0 against 244.8 for cosine: bought nothing here |
VICReg (reg1-vic1) | +2.88% | the best geometry in the table — anisotropy 0.029, decorrelation 0.017 against 0.061–0.070 for the others, rank 253.0 out of 256 — at the highest quality cost of all. The covariance term does what it promises, and that does not turn into lower loss at this budget. |
| additive term α at generation | — | worse than the identity, worse than a random direction (chapter 06). It is what condemned additive inference and motivated the energy mode. |
wm1-sz1, wm1det, wm1n2-sz1, wm1n2z16-sz1 | in the queue | the central hypothesis: WM gain over the floor, and whether SigReg-z holds off collapse without EMA |
wm1n2e-sz1, wm-sobre-dino | in the queue | the pieces of this chapter: learned level-2 encoder and the two-stage pattern |
The right side of the "Outlook" slide lists scales — particles, atoms, molecules, materials, objects, systems — and asks whether the hierarchical JEPA could be a way to build causal models of any complex system. The other slide gives the example: a modern turbine has more than a thousand sensors; an A380, 25 thousand, read 5 thousand times per second.
It is worth saying precisely what that has to do with a 7.4-million-parameter text model. One thing, and it is structural: in our code, the only piece that knows the input is text is the tokenizer. The encoder is a Transformer over a sequence of vectors; the world model is Pred(abstrair groups by blocks; the energy compares vectors. None of those four pieces changes if the sequence comes from sensors instead of tokens — the action would be a controller command instead of a word embedding, and the goal would be a desired state instead of a written goal. That is why the hierarchy was built as a method of the model, and not as a trick in the generation script.
And what is not worth saying: that the current system does this. It has two levels, fixed-size blocks, and the level-2 action is a mean — the simplest version of "intent" that can be measured. A third level is one more loop over abstrair; a variable-size block (sentence boundary, paragraph boundary) is the next serious question. Both come in after the queue answers whether the two levels that exist beat the floor.
A pilot does not plan a flight in terms of aileron angle millisecond by millisecond. They plan in airports, then in routes, then in altitudes — and only at the end do the commands become surface movement. Each level has its own map, its own "where am I", its own "where am I going", and the level above hands the target to the one below. We have two levels of that map: the token and the block. What the chapter shows is that the mechanism for handing the target downward exists and runs; whether it helps is the number the queue is producing.
--freeze-encoder), the SigReg on the states has nothing to move. Collapse of the states becomes impossible — they are fixed — but the predictor can still find the trivial solution of copying Chapters 04 and 05 wait for the numbers from the queue. This one exists so that, when they arrive,
each one lands in a box of the diagram — and not in a loose table.