The corrections
from LeCun

1 The method: failure → passage → flag → measurement

The house rule is to make things work, not to discard them. Each failure of the queue was taken back to LeCun's text with a single question: what does the talk say that we have not done yet? The answer became additive code (no earlier round changes its behaviour) and a new round on the same budget as the others (700 steps, seed 1337, control reg1 at 5.2706).

2 The memory that did not govern

Chapter 11 measured that memory persists: prediction with the memory of the right text gains +0.019 over the memory of another text, even when the folded past is a whole window behind. And chapter 12 measured that the memory map was the only one of twelve on which the rudder did not govern (0.258 against 0.343 free). The hypothesis "with a short prompt the memory is zero" was tested with 320-token prompts and was not confirmed.

The diagnosis came from the persistence table itself, read again. On round wm1n2mem-sz1:

The empty memory is worse than the memory of another text. That is not what "no past" should give: a zero vector should be, at most, neutral. It is the same hole as ζ in chapter 05: in training, the predictor always received the memory folded from the real past (double windows), and never the zero vector — and the zero vector is what it receives when the prompt is shorter than one block. Inference hands it an input from outside the training distribution, the map errs more than with a random past, and a rudder guided by a map that errs does not govern. The 320 tokens of the earlier test were not enough because a folded past of 64 tokens is not the training distribution (256) either.

The correction is the ζ one, transposed: memory-dropout. In half the batches the predictor receives m = 0. It has to work without a past and learn to use the past when it exists — the talk's "combine with the memory" now includes the case in which there is nothing to combine.

The two tables say opposite things, and both are right. The first: the hole closed — the zero memory (0.1770) became neutral, equal to the memory of another text (0.1777), instead of 0.027 worse. And "long" stopped being worse than each half (0.1755 against 0.1756 and 0.1757): no dilution. But the memory gain fell from +0.019 to +0.002: with half the batches without a past, the predictor learned not to depend on the memory, and it almost stopped carrying anything. The second table is what was wanted: on the map that was the only one not to govern, the head with the cost on the real state gives 0.301 against 0.224 free — the rudder governs, and the memory map leaves the list of exceptions (13 of 13 maps). The price came in plausibility (PPL 37 → 58), as on the maps where the rudder governs without lowering PPL. The diagnosis was right: it was the input outside the training distribution, not the memory itself. What remains is the trade-off: 0.5 dropout closes the hole and erases the memory; the fraction that closes the hole and preserves the +0.019 is left for the round with 0.25 — and for the consulted memory of chapter 14, which does not dilute.

3 The level-2 encoder: anticollapse by dimension

Chapter 11 closed the learned level-2 encoder with three shortcuts: shrink (mean + correction: block floor 0.171 → 0.013), lose continuity (SigReg on the abstract states: consecutive blocks orthogonal, cos 0.07, and the worst cost of the queue) and align (fixed norm: every block in the same direction, cos 1.000). The lesson written there: the level-2 anticollapse must fix scale and variance between blocks without destroying the correlation between neighbours.

The talk has the exact taxonomy of this problem. The matrix of a batch — one row per sample, one column per dimension — becomes informative in two ways: all rows different (sample-contrastive methods) or all columns different (dimension-contrastive methods: VICReg, Barlow Twins, and SigReg itself as the limiting case). SigReg-s2 demanded that each block be a sample from an isotropic Gaussian — and independent samples from a Gaussian in 256 dimensions are orthogonal to each other. That was the orthogonality measured. VICReg asks for less: each dimension of the abstract states with a minimum standard deviation (hinge, not quadratic) and the dimensions decorrelated. Nothing forces block k+1 to be independent of block k; it only forces that no dimension dies and that none repeats another. It is the variance term chapter 11 asked for, applied to the batch of abstract states (--vicreg-s2, the same regularizer as round reg1-vic1 of chapter 04, now on s₂ instead of the table).

The second correction comes from the other side of the same talk. The three collapses share a cause: the level-2 encoder is trained by the prediction loss it feeds, on both sides — and the easiest solution to that loss is the target that does not change. The distillation family solves this by construction: the target comes from an encoder that receives no gradient, a moving average of the weights of the encoder that learns (--world-ema). The learned encoder can no longer pull the target towards the prediction, because the target belongs to another model — delayed, but another.

The fourth door closed, and it was the talk's recipe that closed it. The block floor did not shrink: 0.399, more than double the raw mean's 0.171, against 0.013 for the encoder with no anticollapse and 0.032 for VICReg. The cosine between blocks landed at 0.804, inside the healthy 0.5 to 0.9 band — neither orthogonal like SigReg-s2 (0.07) nor aligned like the fixed norm (1.000). And level 2 in one step (0.265) predicts much further than level 1 rolled eight times (0.851): the hierarchy is doing what it promised. The cost to the generator, +1.22%, is the lowest of every level-2 encoder — the others cost 6.2% to 11.3%.

It does not collapse, and it improves with the budget. The block floor nearly doubles, from 0.399 to 0.710: with more training the encoder spreads the blocks more, not less. The norm grows from 16.2 to 22.3 and stays above the raw mean. The probe keeps favouring the encoder at both budgets, and the rudder governs at both, lowering plausibility along with it (PPL 25.4 → 15.4 at the full budget). The shortcut that led five encoders to collapse does not appear even when the model has ten times longer to find it.

The mechanism is what chapter 04 had already described and had never been applied here: the three earlier collapses share one cause, which is the encoder being trained by the loss it itself feeds, on both sides. The easiest solution to that loss is a target that does not move, and shrinking the space is the cheapest way not to move. With the EMA teacher the target comes from another model — delayed, but another — and the encoder can no longer pull it towards itself. No regulariser was needed: SigReg-s2, VICReg and the fixed norm were attempts to forbid collapse; distillation simply removes the incentive.

VICReg did what the taxonomy promised and no more. Continuity between blocks stayed intact — cosine 0.703, between the raw mean's 0.81 and SigReg-s2's 0.07 — and level 2 still predicts further (0.0160 against 0.0180 rolled). But the block floor stopped at 0.032: up from 0.013 and still five times below the raw mean's 0.171. The regularizer normalizes the scale before measuring variance (which chapter 04 did on purpose, because of weight tying) — and so it does not see the shrinkage. The cost to the generator (+8.27%) sits among the other encoders, and the rudder does not govern on this map (0.257 against 0.309 free), as it governed on no level-2 encoder with a shrunken scale. It is the fourth door, and the talk has the sentence that explains it: "every empirical measure of information we have is an upper bound; we come up with a good upper bound and we cross our fingers". A satisfied regularizer does not guarantee information — only the statistic it measures. The missing ruler is the interview's, "information is relative to whoever interprets it": a probe that reads the block from the abstract state, the same for every encoder (model/info_level2.py, section 9).

4 The cost to the generator: the family that scaled

Chapter 05 measured the price of the end-to-end world model with SigReg-z: +9.59% at weight 1.0, +3.25% at weight 0.1; and the stop-grad control, +1.11%, with an almost identical map. Criterion 2 of chapter 05 fired: end-to-end SigReg does not pay for what it costs on this budget. The two-stage round on the frozen DINO gave zero cost and the largest rudder effect of all — but with a worse map (0.665 against 0.150).

The same table gains a control from the talk's other taxonomy: "all the rows different" (sample-contrastive, the family of CLIP and SimCLR) against "all the columns different" (by dimension: VICReg, SigReg). The world model only had the by-dimension side. Round wm1-nce1 swaps SigReg-z for an InfoNCE between the prediction and the target — each prediction close to its target and far from the targets of the other positions in the batch — and measures whether the price to the generator belongs to the family or to anticollapse itself.

The talk is explicit about the two families. SigReg is "conceptually my favourite, but very recent, and we haven't scaled it up". The distillation methods — "you don't propagate gradient through the encoder on the right; it uses an exponential moving average of the weights of the encoder on the left" — are "the ones we scaled up, with really good results for images and video: I-JEPA, V-JEPA". Our stop-grad (wm1det) is half of that recipe: a target without gradient, but from the same encoder at the same step. The other half is the delay: the target comes from a teacher that follows the student with momentum 0.996 — the same teacher the DINO round of chapter 04 already built. If the target changed together with the student, the two could agree on anything; the delay is what gives a stable target without needing SigReg on the states.

Sample-contrastive is the worst of the entire queue, by a wide margin. InfoNCE costs +23.87% to the generator — double the second worst — and delivers a map that looks excellent and is not: MSE 0.084 with cosine 0.97. Those two numbers together are the collapse signature described in chapter 04: the predictor is nearly perfect because the states became nearly identical to each other, and the floor cosine rose along with it, to 0.66. The talk's taxonomy separates contrastive from regularised methods by saying the former "do not scale well with dimension"; here, in 256 dimensions and with 1,024 negatives, it never even competed. It is the negative control §4 needed: without a regulariser forcing variance per dimension, pushing samples apart does not stop the space from shrinking.

The family that scaled is the one that costs least. With the EMA teacher's target, the world model costs +0.45% to the generator — against +1.11% for stop-grad on the same encoder, +3.25% for SigReg-z at weight 0.1 and +9.59% at weight 1.0. It is the lowest price of the whole queue for a map trained together with the generator. The map itself sits slightly below stop-grad (MSE 0.644 against 0.535; cosine 0.74 against 0.79, over similar floors) and well below end-to-end (0.150), and the gain over the floor is the same (+0.81 against +0.87). It did not collapse: the MSE/floor ratio is 0.44 and the floor cosine stayed at 0.50 — the teacher's delay held the encoder without SigReg on the states, as the talk says it held for images and video. Chapter 05's reading closes like this: the world model's cost to the generator was, almost entirely, the price of end-to-end SigReg-z; distillation buys 85% of the stop-grad map for 40% of its cost. This round's modes table says whether that cheaper map still governs.

It governs, and it is the cheapest map that governs — but it separates less. The cost head on the real state takes affinity from 0.301 to 0.341, the same ceiling as the end-to-end map (0.344), and this time lowering plausibility (PPL 76 → 32) instead of raising it. So far the +0.45% map buys what the +9.6% map bought. The rest of the battery shows what the price left out: order violations raise the map's error by 18% and the dependency energy by 36%, against 38% and 130% on the end-to-end map — the EMA map knows less about what a well-formed sequence is, and the guardrail feels it first: the mixed head with γ_c = 3, which zeroed digits on the imagined state in four of four maps, stops here at 16.7% — and not for lack of a head: the football head separates as much as on the other map (interest rates 0.712 against 0.570; code 0.194 against 0.057), and the guardrail head trained to the same accuracy (0.987). What is missing is for the imagined state to be close enough to the real one for the head to recognise it. The distrust rule also flips sign: on the precise map it raises affinity (0.385); here 14% of the tokens fall back to the policy and affinity drops to the free level (0.302), because surprise exceeds 1 more often on a map that errs more. It is the same ruler-and-map pairing as §8 (anticipation): the heading tolerates a weak map; the guardrail and the surprise do not. For the generator, the distillation family is the one that holds; for the guardrail on the imagined state, the map needs the precision that end-to-end SigReg-z bought — and the wm1n2e-ema round (§3) says whether level 2 with an EMA target lands in between.

5 The guardrail on the imagined state

Chapter 12 closed with an uncomfortable empirical rule: on six of seven maps, the "sem-numeros" guardrail only acts when applied to the candidate's real state (the candidate appended to the context, one extra pass). On the states the map imagines — which is where the talk puts the guardrail, "applied to all the steps in the sequence" — almost nothing. The head was trained, like all of them, on real states.

The passage that answers is the same as in chapter 12, read with one more word: "a small head on top of the representation". The representation the planner consults is not z_t+1; it is Pred(z_t, a_t) — a regression towards the mean of the futures, with less detail (which is what JEPA does on purpose). A linear projection fitted on the distribution of real states has no reason to separate candidates on the distribution of imagined states. The correction is to train the head where it will be read: costs.py --predicted (imagined only) and --mixed (real and imagined, so the same head serves both places where the bench applies it).

Three readings, in the order they were measured. The head trained only on imagined states does not solve it (56.2% → 53.1%): the distribution of predicted states is narrower, and a head that has only seen that distribution does not separate well either. The mixed head, with the same weight 1, already acts in both places (43.8% imagined, 12.5% real, against 56.2% and 25.0%) — and with a lower PPL than without the guardrail on the imagined state (7.1 against 8.0). With weight 3, the guardrail on the imagined state alone zeroes the digits, with PPL 21.7: it is the talk's constraint working in the place the talk puts it, without the extra pass of the real state. Weight 10 adds nothing beyond lost plausibility (PPL 25.7) — from 3 on the constraint is already obeyed; the rest is price. The last row is the control: the real-state head with the same weight 3 does not move the imagined state (56.2%, the same as without a guardrail) and improves only the real one (25.0% → 15.6%). On this map the effect belongs to the head, not to the weight: a projection that has never seen the imagined states does not separate candidates on them, however much it weighs.

On the second map both things count. The weight alone already goes half the way (real-state head, γ_c = 3: 46.9% → 13.3% on the imagined state), and the mixed head with weight 1 reaches the same point (12.9%). Only the two together zero it: mixed with γ_c = 3, 0.0% on the imagined state, on two of two maps, with PPL from 19 to 22 — the same price the real state used to charge. On the more precise map (wm1n2-sz1, cos 0.89 per token) the real-state projection still reads something on the imagined state, because the imagined resembles the real; on the weaker map (wm0.1-sz1, cos 0.71) it reads nothing, and only the head that saw both separates. The rule that enters the bench: the constraint is trained where it is read — the mixed head is the default of costs.py for guardrails, and γ_c = 3 the starting weight. The talk's guardrail, applied to every imagined step with no pass on the real state, works.

The "you can't jailbreak it" test

The talk makes a strong claim about this design: a system that can only optimize the objective and the guardrails "can do nothing else" — no prompt takes it out of that, unlike an LLM adjusted by fine-tuning, where "there is always a way to break the conditioning". In text this has a direct test: the prompt asks for what the guardrail forbids. Two prompts, mixed head, γ_c = 3, three seeds; the fraction of digits is measured only on what the model generates.

On both prompts and both maps, the guardrail drops the digits from 35–75% to 0–16%, and "ignore the rule" does not work: the request in the prompt does not enter the energy, only the state it produces does. But the talk's claim is about a system that does nothing but optimize the energy, and ours is not one: the search runs over the k most probable tokens of the generator, then samples at 0.9 among them. When the generator, pushed by the prompt, puts only digits among the candidates, the guardrail picks the least bad — that is what remains on "1, 2, 3," (12.9% and 15.6%). And on wm0.1-sz1, with that prompt, the real state alone failed (43.8%): the candidate "," after "3" has a real state the head does not flag as numeric, and the list goes on. The measured conclusion is more precise than the talk's: the guardrail in the energy resists the prompt in the proportion in which the search belongs to it — and the fraction that still passes is the fraction where the policy (the generator's top-k) decides before the energy does. For "you can't jailbreak it" to hold in full, the search has to run over the energy, not over the generator's short list: that is mpc mode with candidates coming from the latent, not from the policy, and it stays open.

The next test closed that door and opened another. With compass.candidates = latent, the first-step candidates stop being the policy's top-k: a continuous action is optimized against the energy (task + guardrails, with plausibility bounded by the policy's top-64) and the candidates are the k tokens of the whole vocabulary closest to it. Result on the "1, 2, 3," prompt: 0.0% digits on the imagined state alone, on both maps (wm1n2-sz1: 12.9% → 0.0%, PPL 56.9; wm0.1-sz1: 15.6% → 0.0%, PPL 64.2). The search now belongs to the energy, and "you can't jailbreak it" holds in full. The price showed up on another ruler: on the same wm1n2-sz1, the affinity of the best configuration drops from 0.344 to 0.236 — below free (0.292). The tokens closest to an optimized continuous action are good for avoiding (the guardrail is a region not to enter) and bad for going (the heading is a fine direction that the head's linear projection does not pin to one point of the vocabulary). "Both" (the union of the two sets) recovers part of the leak (15.6% on wm0.1): the policy, when it is among the candidates, still wins. The measured rule: candidates from the policy for the heading, from the latent for the constraint — and the two together still have no single candidate set that serves both.

6 What changes in the project

• Training (model/train.py): --world-memory-dropout, --vicreg-s2, --world-ema. All off by default; no earlier round changes.
• Heads (model/costs.py): --predicted and --mixed; the head file records which distribution it was trained on (sobre).
• Modes (model/modes.py): --gamma-cost, the weight of the guardrail heads — the table of chapter 12 used 1.0.
• Laboratory and queue: the three flags and the two head options are on the bench; the four rounds are in geometry_duel.py, with the passage that motivated them in the comment.

7 Researching further: where the unpredictable lives

The next reading of LeCun went after a claim that still had no experiment. The talk says what JEPA does on purpose: "by constructing a representation of Y, the system can eliminate all the information about Y that is simply not predictable" — which is why the prediction becomes "more abstract, with fewer details, but more accurate". In text, "unpredictable" has a ready ruler: the entropy of the generator's own distribution over the next token. If the representation discards the unpredictable, the map's error should grow less than the identity floor as entropy rises (model/unpredictable.py: 64 windows of 128 tokens, 8,128 positions, by entropy quartile).

The result is not what the question expected, and it is better than that. The map's error is flat: on wm1-sz1, 0.143 in the most predictable quartile and 0.146 in the most unpredictable, with zero correlation with entropy; on wm1det, the same. The floor does not grow with entropy either. The reason lies in the definition of action: in text, the action is the token that came, and the map predicts the next state given the token. All the uncertainty of "what comes next" sits in the choice of token — which is the distribution the generator already models — and what remains, the state transition given the token, is almost deterministic, as predictable at an entropy of 8 as at an entropy of 3. The talk describes JEPA eliminating the unpredictable from the representation; here, the architecture eliminated the unpredictable from the prediction by construction, because it conditioned it on the action. It is the structural reason why the latent ζ of chapter 05 found nothing to carry once ζ-dropout closed the leak: at level 1 there are no "multiple futures" given the token; the map does not need ζ. Where ζ may be worth something is at level 2, where the action is the mean of a whole block and the choice of tokens inside it is left out.

The weaker map tells the other half. On wm0.1-sz1 (weight 0.1, cos 0.71) the error rises with entropy (0.543 → 0.720, Spearman +0.23) and follows the generator's surprise on the real token (+0.47): an under-trained map still pays for the unpredictable, and what end-to-end training buys, with its cost to the generator, is exactly the transition that does not depend on entropy. It is a new ruler for "the map is ready": the error stops depending on where the text is hard.

8 Two more doubts, with numbers

Does the "emotion" anticipate?

The interview: the systems "will have emotions, because they are anticipations of outcome — if they predict they will fulfil the goal, that makes them happy". The bench records, at each token, the lowest energy the planner found (energia_prevista, chapter 12). Nobody had measured whether that anticipation anticipates. The test (model/anticipation.py): 12 prompts × 3 seeds with the rudder ("futebol10" head + real state, γ = 0.1); for each generation, the mean predicted energy and the real outcome — the affinity by the independent judge.

On the precise map, the anticipation anticipates: the generations in which the planner expected to fulfil the heading fulfilled it almost twice as much (0.414 against 0.216), and the correlation is strong. The predicted energy can be read as confidence — and that is what the panel shows. On the weak map, no: the correlation vanishes, and the predicted energy is an internal number with no predictive value. The interview's "emotion" is a property of the map, not of the planner: only a world model that is right can anticipate. The third column says something the interview does not: where the rudder governed most, the map was most surprised (+0.69). Governing is taking the text where the world model did not expect — the price in surprise is the mark of the rudder acting, and the "adjust the map" signal of chapter 09 fires precisely on the generations that worked.

Does ζ carry multiple futures at any level?

Section 7 concluded that at level 1, given the token, the transition is deterministic, and conjectured that the latent ζ can only matter at level 2, where the action is the block mean. The test (model/futures.py, round wm1n2z16zd0.5-sz1, the one with the closed channel): per level, the error with ζ = 0, with one sample of ζ, the best of 8 samples, and the dispersion among the 8 futures — real futures spread in the order of the mean future's error; noise spreads much less.

The conjecture of section 7 was not confirmed. At both levels, the eight sampled futures spread 15–18% of the mean future's error: ζ trembles around the mean, and "best of 8" gains only what selection among noisy samples gains. Level 2 shows no more choice than level 1. At this scale (7.4 M, blocks of 8, a 200 MB corpus) the next block's mean is predictable enough from the mean of the actions that no mode of variation is left for ζ to carry. The contrast with the leaked channel shows what high dispersion means when it is bad: 0.84 of the error, but the single sample is worse than the mean future — different futures that are not plausible futures. The talk's latent variable stays on record as a piece this budget does not need: it is worth something where the action leaves a mode of variation out, and blocks of 8 text tokens do not yet. Larger blocks (a sentence) are the next ruler.

9 The information of level 2, measured by an interpreter

Four anticollapses, four doors. The search in the text of the talk (tools/transcripts.py, sentences 264–275) has the explanation: "we don't actually have objective measures of information content, because all the proper definitions are based on knowing the distribution, and we only have samples coming out of an encoder"; "to maximize something you would need a lower bound; the problem is every empirical measure we have is an upper bound. So we come up with a good upper bound and we cross our fingers". SigReg, VICReg and the fixed norm are upper bounds: satisfying them does not force the state to carry the block. And the interview (sentences 179–185) says what the ruler would be: "the amount of information in a message is not an absolute quantity; it depends on the person interpreting it". The honest measure is a fixed interpreter, the same for every encoder: a linear regression that tries to read, from the abstract state, the mean of the embeddings of the tokens of the block itself (reconstruction) and of the next block (prediction) — model/info_level2.py, out-of-sample R², next to the same R² for the raw mean of the same round.

The interpreter confirms it, and shows that the solution that looked obvious was not the best one. The encoder with the EMA target is the only one of the six learned encoders that does not lose information: it reconstructs its own block as well as the raw mean (0.712 against 0.714) and anticipates the next block better than it does (0.223 against 0.210). The norm grew instead of shrinking (16.2 against 13.5) and continuity rose (0.820 against 0.745). By the ruler the interview asked for — "information is relative to who interprets" — this is the first level-2 encoder worth having.

And the piece that looked like the direct answer failed. Putting the probe itself inside training (--world-enc2-probe), optimising the encoder to be readable, gave the worst result in the table: 0.059 R² on the next block against 0.274 for the same round's raw mean, with the norm crashing from 11.4 to 1.554 and the cosine to 0.361. Optimising the metric destroyed what it measured. It is Goodhart's lesson applied to an anticollapse, and it is worth recording beside the success: measuring with an interpreter works; training against the interpreter does not. The talk warns that every information measure is an upper bound — using it as a target is choosing the shortcut the upper bound cannot see.

Read by the interpreter, the picture has no ambiguity left: all five learned level-2 encoders lost information, every one, relative to the raw mean of their own round — from 0.78 to 0.30–0.76 on reconstructing the block, and from 0.24 to −0.26–0.22 on anticipating the next block. No regularizer prevented it. SigReg-s2 preserved reconstruction (0.756) but lost half of the anticipation (0.116) — Gaussian, independent blocks carry their own block and nothing of the next, which is what the orthogonality (cos 0.10) already said. The fixed norm lost the least in anticipation (0.221), but takes every block to the same direction (cos 1.000), which the linear interpreter can still read and the level-2 predictor cannot. VICReg, the only one that kept continuity, lost the most (negative R²: the abstract state anticipates the next block worse than the constant mean). And the last column says why the horizon test of chapter 11 misled: "level 2 predicts further" was true on each encoder's own ruler, but the ruler was shrinking with it — the probe, which does not see scale, shows that what was being predicted was less and less. The raw mean, which learns nothing, is the only abstraction that carries the whole block and a quarter of the next.

10 What the searches became in code

The 74 searches of chapter 10 §8 were audited with one question: does the passage ask for an algorithm that does not exist here yet? Seven did. Each became an additive piece — a flag, a bench parameter or a script — and, when it requires training, a round in the queue. Nothing that existed changed its behaviour.

Three passages were audited and did not become code, with the reason: "relative information" at level 1 already has an interpreter (the token head and the cross-entropy are the interpreter of the state z); "the cost is used for inference, not for learning" is already how the heads work; and "consciousness", "retina", "Stargate" are context. The rule that stays: a passage becomes code when there is a measurement that shows whether it holds.

11 Six passages, six algorithms — and what each one gave

The rule of this section is the one that holds for the whole chapter: a passage from LeCun is not applied until there is an algorithm it describes and a measurement that shows whether it holds. Six passages sat in the table of chapter 10 as justification for something that already existed. Each became a new piece, measured on the wm1n2-sz1 map.

1. "A dependency, not a function" → dependency energy

The talk (#279–280): between x and y "there's no single y for a given x; it's a relation, not a function", and the way to capture it is an energy that is low on compatible pairs. Our map is a function — Pred(z_t, a_t) — because it conditions on the action. The dependency appears when the action is not given: from z_t come as many states as plausible actions. The piece is the energy of a target state as the minimum, over the k most probable actions of the policy, of the prediction error: E(z_t, z′) = min_{a ∈ top-k} ‖Pred(z_t, e_a) − z′‖² — low on every state some action explains, high on the one no action explains (Transformer.energia_dependencia). It is the talk's landscape with one valley per action. The ruler is chapter 05's violation of expectation, now independent of the token that actually came:

It works, and it separates more. Given the token, destroyed order raises the error by 38%; without giving the token, the dependency energy rises 130–142%: none of the eight plausible actions leads to that state. The document swap, which is semantically plausible, rises little on both rulers (+30% and +34%) — the dependency distinguishes a violation of the world (order, sequence) from a violation of topic. It is the common-sense ruler chapter 05 did not have: what the talk calls "the error shoots up on the impossible" becomes three times sharper when the question is "does any action explain this?" instead of "does the action that came explain this?".

2. "The baby throws the toys to see" → active experimentation; 6. "The EMA came to stabilize variance" → stabilized target

The correction loop of chapter 09 trained the map on every collected trajectory, with the model's own moving target. Two pieces: --active corrects only on the sequences where surprise/floor was above the median — the experiment is where the expectation failed —, and --target-ema supplies the targets from an EMA copy (the original idea of the EMA, "stabilizing variance" in DeepMind's RL, #420), so the corrected map does not chase its own target.

No harm, and no generalization. The same as the full correction of chapter 09 (0.161 → 0.159 on test): the map memorizes the trajectories it saw and does not change on what it did not see. Choosing only the highest-surprise sequences did not change that, nor did the stabilized target. The honest reading: on this budget the surprise is not systematic — the map errs in different places on every text —, and correcting where it erred does not teach where it will err. The talk says to use RL "only when the plan does not yield the predicted outcome"; the piece is ready for when there is a systematic error (a new domain, a style), which is where it has something to correct.

3. "I don't know how long the taxi takes" → persistent subgoal with arrival

The level-2 plan of chapter 09 was remade at every token. The talk (#151–163) says the high-level plan does not know how long each step takes — it says where to arrive. The piece: the subgoal stays fixed until the state gets close to it (cosine ≥ threshold) or patience runs out (compass.subgoal_until).

It got worse, and the ruler was half the cause. With cosine 0.9 as arrival and the token ruler, the state never arrives: in 16 subgoals, zero arrivals, mean cosine 0.41 — a subgoal is an abstract block direction, and a token's state does not reach it — so the plan goes stale for the 8 tokens of patience and level 1 chases a target that no longer matches the context. The fix is to measure arrival with the same ruler that produced the subgoal: the abstract state of the current block (abstrair over the last 8 states) against the subgoal. With it the state does arrive (mean cosine 0.58; at 0.9, 3 of 19 arrive, at 0.7, 65 of 74, at 0.5, 95 of 101) and affinity returns to 0.245 with PPL 18.0 — exactly replanning at every token, no more and no less. What the right ruler bought was to stop getting worse; what persistence did not buy was heading: a subgoal held for 8 tokens is worth 0.03 less than one remade at every token (0.216 at 0.9), and one held until arrival is worth the same. The reading is the one from the 10 ms passage (section 10): in a block of 8 tokens, level 2's "where to arrive" sits one or two tokens from the current state — you arrive before persistence has anything to persist. The passage is right about what the high-level plan does not know; for persistence to pay, the level-2 step must be longer than the patience — that is the --world-block 32 round in the section 10 table, which the block ruler can now measure. The block ruler becomes the bench default.

4. "A differentiable measure of information" → the probe in training

It is the piece of §9, --world-enc2-probe: round sd1 is in the queue and the table of §9 receives its row when it closes.

5. "Is the door open? — two samples" → prototype cost

The talk (#487–491) says a task cost "basically requires two samples". The literal piece: no regression, the direction between the mean states of one positive sentence and one negative, threshold in the middle (costs.py --prototype).

Two samples are not enough in this space. The prototype puts Python code (0.680) above football (0.471): the direction between two sentences carries what distinguishes them and everything else in which they differ. As a cost, it does not govern (0.284 against 0.292 free). The talk speaks of video representations where "the door is open" is a clean direction; on a 7.4 M encoder trained on text, the clean direction needs ten sentences and a regression that discounts the rest. The piece stays on the bench as the "how many samples" control — the number, here, is ten, not two.

And the two that §10 left under test

Adaptive candidates (policy for the heading; latent only when the policy's candidates already violate the guardrail): on the "1, 2, 3," prompt, 0.0% digits on the imagined state alone on both maps (wm1n2-sz1 and wm0.1-sz1) — the same as the latent candidates —, and the heading affinity stays at 0.344, equal to the policy's. The rule "policy candidates for the heading, latent for the constraint" now has a single candidate set that serves both, and "you can't jailbreak it" holds at no cost in heading. Distrusting the map (confiar_se_surpresa = 1: the token after a surprise above the floor stays with the policy): affinity 0.385 against 0.344, with PPL 85 against 70 and 86% of tokens planned — more heading with less planning, at the price of plausibility. The passage "RL only when the plan fails" has an inference-time version: do not correct the map, just do not trust it on the next step.

12 Three passages only a training run can answer

The six algorithms of section 11 fit inside inference: change the planner or the ruler and measure. Three passages from LeCun do not fit — they ask for a different training objective, and so only a round can answer them. Here are the three questions and the three tables, written before the numbers. The cells are filled when each round closes.

12.1 Mask, and predict the representation of what was covered

The talk states JEPA's recipe for text without hedging: "corrupt or mask the input and predict the representation of the original, not the input". Our world model predicts the next state; it never predicts a state covered in the middle. The piece is --world-mask p: with probability p, a block in the middle of the window is swapped for a learned mask vector, and the state at the masked positions must predict the state that would be there — the same map loss, with the target in the middle instead of ahead. It is I-JEPA applied to text, and the question is whether a target in the middle teaches something the target ahead does not.

The mask does learn, and costs the same. Predicting the covered middle block gives error 0.093 against a floor of 0.114 — a 19% gain over copying the previous block. It is not the forward map's gain (0.167 against 1.072, nearly sevenfold), but it is clearly non-zero: a target in the middle of the window teaches something. And the price to the generator is +8.79%, indistinguishable from the same map's +9.55% without the mask under §12.3's noise ruler. The piece comes for free.

What it has not shown yet is what it is for. A 19% gain over the floor is a training number; the question is whether the masked map governs better, detects violations better, or closes the guardrail better. That is the battery, and it answers in this table's missing column.

12.2 Sentence-sized blocks: "you can't plan in 10 ms actions"

The talk dismisses planning at the finest step: "you cannot plan in terms of ten-millisecond actions — it makes no sense; that is why planning is hierarchical". Section 11 measured the same thing from the other side: with blocks of 8 tokens, the level-2 subgoal sits one or two tokens away, you arrive before persistence is worth anything, and holding the plan yields nothing over remaking it. A block of 8 is our "10 ms". The round swaps the block for 32 tokens — a sentence — with the ζ latent on, which is where the block's uncertainty should show up.

Both rounds were re-run on the same card on 17/09 (the block-8 one had been overwritten by the block-32 one on 11/09; the block-32 one repeated its numbers from then to the fourth decimal). Both questions came back with the same answer: no.

• ζ did not start choosing between futures. The spread across eight sampled futures, as a fraction of the mean future's error, stayed at 17% on level 1 and fell from 16% to 12% on level 2. The gain from picking the best of eight samples is similar for both blocks (26% and 29% below a single sample on level 2). With a sentence per block, ζ still trembles around the mean.
• The subgoal is still close. Consecutive 32-token blocks are already very similar to each other: the cosine between a block and the next is 0.879, against 0.781 for blocks of 8. With an arrival threshold of 0.7, below that similarity, the subgoal is reached in 1.4 tokens on average (259 arrivals out of 266). At 0.9 it arrives 31 times out of 43, every 9 tokens. With blocks of 8, the same 0.9 arrives only 2 times out of 50. The distance grew, but measured by the block cosine the subgoal did not move far away.
• The cost is the same. +7.85% against +8.40%: half a point, within the seed spread of §12.3. Level 2 with 32-token blocks still predicts better than copying (0.052 against 0.076), but rolling level 1 for 8 tokens is worse than copying (0.099), because 8 tokens do not reach the next block. The heading steers on both (affinity 0.392 and 0.382 against 0.275 free).

The reading moves: the arrival ruler needs a threshold above the natural similarity between consecutive blocks, not bigger blocks. With a threshold already reached by copying the current block, "arriving" measures no plan at all.

12.3 Seeds: "slightly different brains"

The interview, on why two systems trained alike are not the same system: "the brains are slightly different, and diversity is the strength of the tribe". This entire collection's queue ran on one seed (1337). Every 0.5% difference we have read so far could be initialisation noise, and there is no way to know without repeating. The round repeats the synthesis (reg1-lat1o) with seeds 2 and 3, and the report now aggregates mean ± standard deviation per round name.

The spread between seeds is 0.038, and that changes how several sentences in this collection should be read. Three identical training runs, changing only the number that initialises the weights, give validation 5.2233, 5.2867 and 5.2924. The gap between the best and worst seed is 0.069 — larger than the gap between many rounds this collection compared as if they were distinct. This queue's control gave 5.2577. These are two measurements of the same control: 5.2706 is from the original duel run, used as the baseline in chapters 03 to 13; 5.2577 is the one recorded in the run's current training history, used from here on and in chapters 15 and 16. The gap, 0.013, is smaller than the seed spread measured in this section — by the ruler below, both baselines give the same reading. The synthesis reg1-lat1o, which chapter 04 recorded as winning 0.65% over the control, averages 5.2675 against 5.2577: +0.19%, inside the noise. The gain did not vanish because it was wrong; it never had an error bar, and now it does.

The ruler that comes out of this applies to everything written before: a validation difference below ~0.04 (0.75%) is not a result. By the new ruler, the EMA target (−0.33% against the control) and stop-grad (+0.14%) are tied with the control and with each other — chapter 13 §4 ranked them, and the ranking does not hold. What still stands is everything separated by wide margins: end-to-end SigReg-z (+9.6%), InfoNCE (+23.9%), the hierarchy (+8.8%). What falls are the technical ties that were read as victories.

The deviation is the same in all of them: 0.84% of the loss, four times out of four. That turns the finding from "one round was noisy" into a ruler: any difference below ~0.9% is not a result, and any above ~2.5% (three deviations) is. The synthesis reg1-lat1o, which chapter 04 recorded as +0.65%, now measures −0.13% against the control: tied, and the sign even flipped. The two hierarchies, by contrast, are separated by 0.37 of loss — eight deviations — and that ordering holds: the EMA target costs 1.15% where SigReg-z costs 8.17%.

The interview had said this by another road: "the brains are slightly different, and diversity is the strength of the tribe". The line is about there being no canonical brain. The experimental translation is that a single round is not a measurement — it is one sample from a distribution whose width nobody had measured. It cost three minutes of GPU to find out the width is 0.038.