Antivision · Latent Rehearsal
10

What JEPA is,
according to LeCun

Two transcripts — the talk on world models and the interview — read concept by concept, with the project file where each one applies, what was already right and what was corrected because of them.

Chapter 10 Prerequisite: 09 (the hierarchical architecture)

1 The two sources

The first source is Yann LeCun's talk on world models, the technical one: the world model, inference by energy, collapse and how to avoid it, SigReg, hierarchy, DINO as encoder, V-JEPA and expectation violation. The second is an interview with LeCun, less technical and more precise on three points that the talk leaves implicit: what JEPA is in relation to the Transformer, why predicting the world is impossible (and in which space it stops being so), and what reasoning is. This chapter exists because these three points change the reading of what we built.

2 JEPA is not an alternative to the Transformer

The audience member's question was exactly that — "are you proposing an alternative to the Transformer?" — and the answer is direct: JEPA is a macro-architecture, an arrangement of modules; the modules can be Transformers. What it replaces is the objective of current models — the autoregressive architecture that predicts the next symbol in data space —, not the piece that computes representations.

Checked against our code

In model/architecture.py, the Transformer class is the encoder: it receives tokens, returns states z. The predictor is mundo (and mundo2 at level 2), which receives (zt, at) and returns zt+1. JEPA is the arrangement of the two with the cost in latent space. The two variants of chapter 02 — classic and modern — are the same module with different parts; the choice between them is orthogonal to JEPA, and the architecture duel remains fully valid inside it.

What the interview calls "GPT" — the autoregressive objective — is also here, in the token head. Chapter 09 explains why it stays: it is the decoder, the guardrail and the ruler. JEPA replaces its role, not the piece.

3 "Predicting the world is impossible" — where, exactly

The interview's argument is finer than the slogan. Predicting the next frame of a video does not work because representing uncertainty about the future in a continuous high-dimensional space is mathematically intractable: nobody knows how to write a distribution over all possible frames. In text, the problem does not exist: the space is discrete and finite, and the distribution over the next token is a softmax. In his words, "it is possible in discrete space like text".

This has two consequences for us, and both are in the code.

  1. The token head is not "wrong" for text. The intractability that motivates JEPA in video does not apply to our domain. That is why it stays — and that is why §1 of chapter 09 calls the system hybrid without embarrassment.
  2. What JEPA buys in text is something else: abstraction (not spending capacity on unpredictable detail — the camera that turns and reveals a plant whose species nobody predicts) and, above all, a space in which to plan. In text, the unpredictable detail is the lexical choice: "fez o gol" or "marcou o gol". The world model predicts the next state without committing to the word — and it is in that state that the heading is measured.

That is also why the latent variable ζ (--world-latent) is optional here and mandatory in video: the blur of the average of futures exists in both cases, but in text the softmax already carries the uncertainty. Our ζ serves level 2, where a block of eight tokens can continue in many ways.

4 Reasoning is searching in mental space, not in output space

The cube passage: rotate an imaginary cube 90° and say whether it looks the same. Nobody answers by generating drawings and comparing; one answers by manipulating a mental model. And the criticism of LLMs with "reasoning": they generate many token sequences and a second model picks the best one — "primitive", "not how we think", and expensive.

That sentence catches our energy mode red-handed. The beam search of chapter 09 does imagine in the latent, yes — but it branches by tokens: at each imagined step, it enumerates the plausible actions as tokens and expands. It is the miniature version of "generate many sequences and choose". It works, it is measured, and it is not the concept.

Applied because of this reading

The mpc mode, in server/ours.py: a sequence of H continuous actions e1…eH is found by gradient descent through the world model, minimizing

E(e1..H) = task(zH) + γ · (1/H) Σt guard(et | zt),  zt+1 = Pred(zt, et)

No token is enumerated along the way: the search happens entirely in the space of continuous actions. The per-step guardrail keeps each et close to some plausible action in that imagined state — the talk's safety constraint, applied "to every step of the sequence". Only the first action is committed (the token closest to e1) and at the next token everything is replanned: receding horizon, the classic predictive control that the talk dates to the 1960s.

The three modes sit side by side, selectable in the «Antivision · rudder» panel: energy (search by tokens, the "primitive" version), gradient (one continuous action, one step) and mpc (the whole sequence). Nothing was removed — the comparison between them is an experiment, not an opinion.

In the smoke test, mpc with horizon 3 and 8 gradient steps costs about 0.8 s per token on this CPU; the beam search, 0.05 s. The energies of the two modes are not comparable in scale (the beam accumulates −log p; the MPC normalizes the guardrail by the horizon): one compares the generated text and the surprise, not the energy number.

5 Emotion is anticipation of the outcome

The interview defines emotion in a way that can be implemented: systems that optimize an objective with a world model anticipate whether they will meet it, and that anticipation — meeting it or not — is the minimal form of an emotion. No hardwired anger or jealousy; only the prediction of the outcome.

In our system that number already existed and was not stored: it is the lowest energy the planner found before emitting each token. Now it goes into the trajectory (energia_prevista, in /nosso/trajetoria) and into the panel, next to the surprise. The two series together tell the story of a generation: what the system expected (predicted energy, low = "I will keep the heading") and what happened (surprise, high = "the world went where I did not predict"). The distance between the two is the material of the correction loop of chapter 09.

6 Persistent memory

LeCun lists, in both talks, four characteristics of an intelligent system: understanding the world, having persistent memory, reasoning, planning. The talk is explicit: perception gives only the current state; to have the complete state of the world, "you have to combine that with the content of a memory". We do not have that. The only state of our system is the context window — 256 tokens in mini — and what leaves it ceases to exist. It is the clearest gap between what the talks describe and what is built, and it is recorded as such in the table of §7, with no piece, no flag and no deadline.

6b Three more pieces, from the second reading

A front-to-back reading of the talk, rather than a keyword search, found three concepts stated in so many words and with no piece in the project. They are implemented; the data for each depends on the wm* runs.

7 The map: concept → file

Each concept from the two transcripts, the project piece that implements it, and the status. "Already there" = checked, no change. "Applied (10)" = done because of this reading.
Concept (source)Piece / fileStatus
JEPA is a macro-architecture; the Transformer is a module inside it (interview)architecture.py: Transformer = encoder, mundo = predictor; note in chapter 02Already there; documented (10)
Predicting in data space is intractable in the continuous case, tractable in text (interview)token head kept, perdas(); §3 of this chapter; 09 §1Already there; justification corrected (10)
Predict in representation space, not in data space (talk)perdas_mundo(): MSE of Pred(z,a) against zt+1, with identity floorAlready there; wm* runs in the queue
Perception → state → proposed action → world model → energy; zero energy when met (talk)ours.py: _custo_tarefa = 1 − cos (zero at the goal)Already there
Inference by energy minimization, more powerful than fixed propagation (talk)energy, gradient, mpc modesAlready there; mpc applied (10)
Reasoning in mental space, not by generating many outputs (interview)mpc mode (no token enumeration); energy mode marked as the "primitive" versionApplied (10)
Guardrails applied to every step of the sequence (talk)_planejar: accumulated −log p; _mpc: distance to the plausible action per stepAlready there; extended to mpc (10)
Hierarchical planning with subgoals (both)mundo2, abstrair(), _subobjetivo; --world-levels 2, --world-enc2Already there (2 levels; "open problem" in the talk)
Collapse: encoder ignores the input, constant representation (talk)identity floor in every metric; --sigreg-z; chapters 03 and 06Already there
Regularized instead of contrastive; minimize the low-energy volume (talk)regularizers.py: cosine, SigReg, VICReg, Barlow; no negative pairAlready there, measured
SigReg: isotropic Gaussian distribution via 1-D projections (talk)sigreg(): Epps-Pulley on 64 projections; --sigreg, --sigreg-zAlready there, measured (in the table: +1.51%, no gain in rank)
Latent variable against the blur of the average of futures (talk)--world-latent, ζ ~ N(0, I), sampled in the plannerAlready there; wm1n2z16-sz1 run in the queue
DINO as encoder + world model + planning (talk)train.py --base … --freeze-encoder; wm-sobre-dino runAlready there; run not yet launched
Expectation violation: prediction error spikes on the impossible (talk)surprise.py, Transformer.surpresa(), /nosso/trajetoriaAlready there
MPC; RL only when the plan fails, to adjust the world model (talk)correct_world.py (smoke test: 0.907 → 0.719)Already there
Emotion = anticipation of the outcome (interview)energia_prevista in the trajectory and in the panelApplied (10)
Persistent memory, combined with perception to form the state (both talks)--world-memory: a vector m folded block by block from what leaves the window, read by the world model; ganho_memoria; memory.pyApplied (10); run wm1n2mem-sz1 queued
Cost and guardrail are "a small projection, trained with very few samples" (talk, answer to the MPC engineer)costs.py: logistic regression on z, type task (E = 1 − p) or guardrail (E += p at every imagined step); compass.cost, compass.guardrailsApplied (10); smoke test: 3 sentences, accuracy 0.99
Abstraction exists to predict farther (talk)horizon.py: level 1 rolled 1/2/4/8 steps × level 2 in one step, on the abstract scaleApplied (10); no data until the wm* runs close
"Standing up from a chair needs no planning; it can be just a policy" (talk)compass.plan_if_entropy: below the threshold the token head decides alone; fracao_planejada in the trajectoryApplied (10)
Combine perception with memory — including when there is no memory (both; third reading)--world-memory-dropout: half the batches with m = 0; round wm1n2mem-sz1-md0.5Applied (13): the hole closed and the rudder governs on the memory map; the memory almost stopped carrying (+0.002)
Anticollapse by dimension: "all the columns different" (talk; third reading)--vicreg-s2: variance + covariance on the abstract states; round wm1n2e-sz1-vs1Applied (13): no direction collapse (cosine between blocks 0.703), but the scale shrank 5× and information was lost (probe −0.257); the heading does not steer
The distillation family (EMA, no gradient through the target) is the one that scaled: I-JEPA, V-JEPA (talk; third reading)--world-ema: targets zt+1 and abstract block from the EMA teacher; rounds wm1ema, wm1n2e-emaApplied (13): wm1ema costs +0.45%, tied with the control across seeds, and steers; wm1n2e-ema is the first learned level-2 encoder that does not collapse (floor 0.399, probe 0.712 against 0.714 for the raw mean)
The constraint is "a small head on top of the representation" — the one the planner consults (talk; third reading)costs.py --mixed: head trained on real and imagined states; modes.py --gamma-costApplied and measured (13): 0.0% digits on the imagined state alone, on 4 of 4 maps
"You can't jailbreak a system that only optimizes the guardrail" (talk)modes.py --prompt-guardrail (prompts asking for numbers); compass.candidates = latent (candidates from the energy, not the policy)Measured (13): 35–75% → 0–16%; the fraction that passes is the policy's top-k; with latent candidates, 0.0% on both maps — at the cost of the heading (affinity below free)
The representation eliminates what is not predictable (talk)unpredictable.py: map error by quartile of the generator's entropyMeasured (13 §7): flat error — given the token, the transition is deterministic; the unpredictable lives in the action
Persistent memory that is consulted and survives the session (both)--world-bank: bank of abstract states + attention; server/bank.py with turbovec (4-bit TurboQuant) on diskApplied (14); round wm1n2bco-sz1 in the queue
Emotion = anticipation of the outcome (interview) — does it anticipate?anticipation.py: predicted energy × real affinity by the judgeMeasured (13 §8): Spearman −0.61 on the precise map; +0.11 on the weak one — the emotion is a property of the map
Latent variable against the blur of the mean — at which level? (talk)futures.py: ζ = 0, sample, best of K, dispersion, per levelMeasured (13 §8): noise around the mean at both levels (dispersion 15–18% of the error); blocks of 8 leave no mode of variation out
Redundancy in the data is a condition of self-supervised learning, "but not too much" (talk)redundancy.py, prepare_data.py --no-duplicatesMeasured (13 §10): 0.28% duplicated, 7.29% repeated 8-grams
Learning new tasks from few examples; "intelligence is what you do when you don't know" (talk)—Out of scope for this cycle
What the reading actually changed

Three things, all additive: the mpc mode, the predicted energy in the trajectory, and the justification for keeping the token head — which went from "it is the measuring instrument" to "in text the autoregressive objective does not suffer from the intractability that motivates JEPA; what it buys here is the space in which to plan". The architecture itself was aligned; what was misaligned was an inference mode presented as the concept when it was the version the concept criticizes.

8 The searches in the text

The transcripts are a single paragraph; tools/transcripts.py cuts them into sentences and returns the ones matching a term, with their neighbours. Each search below was made for a doubt of the project, and the right-hand column says what the sentence changed.

Twenty searches (11/09). "#" is the sentence index in the transcript; talk = concept.txt, interview = concept2.txt.
TermSentence(s)What it saysWhat changed in the project
collapsetalk #245–253"the system can ignore the input and produce constant representations; the prediction problem becomes trivial"; "the whole trick is how you prevent collapse"identity floor on every metric (06); the three level-2 collapses (11)
rows|columnstalk #313–320"all the rows different (sample-contrastive) or all the columns different (dimension-contrastive)"--vicreg-s2 (13 §3) and the --nce-z control (13 §4)
upper boundtalk #268–275"we only have samples coming out of an encoder"; "every empirical measure is an upper bound; we cross our fingers"the interpreter probe info_level2.py and --world-enc2-probe (13 §9)
relative|interpretinterview #179–185"the amount of information depends on who interprets it; entropy is our ignorance"same; and the generator's own entropy as the ruler of the unpredictable (13 §7)
latenttalk #214"blurry prediction because it predicts the average of everything that can happen; you correct that with latent variables"ζ, ζ-dropout (05); futures.py: no multiple futures in blocks of 8 (13 §8)
memorytalk #134; interview #41–43"combine with the content of a memory"; "persistent memory of the type that we have"summed memory (11), memory-dropout (13 §2), consulted bank (14)
distill|moving averagetalk #411–420"the distillation methods are the ones we scaled; the encoder on the right uses a moving average of the weights"; "an idea that came from stabilizing variance in RL"--world-ema (13 §4)
guard|jailbreaktalk #138–146"intrinsically safe: it can do nothing but optimize the guardrail and the task"; "applied to all the steps in the sequence"mixed head, γc, adversarial prompts, latent candidates (13 §5)
hierarch|sub goaltalk #146–170; interview #264–281"sub-goals down to an action that is just a policy"; "we have no idea how to do this"level 2 + subgoal (09), entropy gating (12)
policy|stand uptalk #168–169; interview #278–279"standing up from your chair without thinking: that could be just a policy"plan_if_entropy (12)
redundantalk #85–89"without redundancy you can't learn anything; but not too much"redundancy.py: 0.28% duplicated / 7.29% repeated 8-grams; --no-duplicates (13 §10)
reinforcementtalk #468; interview #78–91"RL is what you do when you're desperate"; "part of the solution, not sufficient"correct_world.py: only when the plan fails (09)
searchinterview #234–238"all reasoning is search in a space of solutions (the shortest circuit)"energy/mpc modes (12); cost per token
uncertain|distribution overtalk #191; interview #116–117, #423–425"in text you can put a distribution over the dictionary and handle the uncertainty"the token head stays; the map predicts the transition given the token (13 §7)
cost functiontalk #102–103"the objective is a cost function: not used for learning, used for inference"cost heads only at inference; the map is not trained by them (12)
common sensetalk #71, #441–453"violation of expectation is how you measure whether a baby learned a concept"; "the prediction error shoots up on the impossible"surprise.py (05, 09)
level of abstractiontalk #333–344"every level of the hierarchy ignores detail of the one below to predict further"horizon.py (11) — and the caveat of the shrinking ruler (13 §9)
three-dimensional|depthtalk #50–56, #456"the baby learns the world is 3D just by watching; a head on the representation predicts depth"linear probes on z (probes.py) and on s2 (13 §9)
fixed number of layerstalk #93–108"inference by search is intrinsically more powerful than a fixed number of layers"the rudder as search; s/token measured per mode (12)
push the energy up|volumetalk #294–304"contrastive: generate points outside and push the energy up; regularized: minimize the volume of low energy"04 (lineage); the InfoNCE control (13 §4)
head on top|projectortalk #234, #440, #487"a small head on top of the representation, trained with very few samples — one projector per constraint"costs.py, mixed head (12, 13 §5)
two views|masktalk #242–245, #440"two views of the same scene, or corrupt/mask and predict the representation of the original"--world-mask: the masked objective, I-JEPA in text (13 §10)
specialized|adaptivetalk #36–41, #75"intelligence is learning a new task quickly; AGI makes no sense as a phrase"heads with few examples (12); the rest out of scope
one GPU|scale uptalk #404–407"we trained world models with SigReg on one GPU; it works decently; what's missing is scale"the same honest scale this project declares (07)

Fifty more searches

The last reading round: fifty terms, each with the matching sentence and what it changes — or does not change — in the project. Where the right-hand column says "no change", the passage is context and is recorded as such.

Fifty searches (11/09, second batch). Sentence indices from tools/transcripts.py.
TermSentence(s)What it saysWhat changed in the project
baby|infant|months oldtalk #46–56at two months the baby only gesticulates and learns a dynamical model of its own limbs; it learns the world is 3D just by being movedcorrect_world.py --active: correction only where the expectation failed (13 §11: does not generalize on this budget)
object permanence|rigidity|stabilitytalk #59–60object permanence, stability and rigidity are learned very earlylow-level concepts learned before physics: the hierarchy is also a learning order
gravity|inertia|intuitive physicstalk #61–63; interview #121–126gravity and inertia take nine months; the child runs the experimentsame: --active (13 §11)
throw|toys|high chairtalk #62, #451the baby throws the toys to see; the ball that vanishes makes the error shoot upviolation of expectation (`surprise.py`): shuffled and inverted continuations are the vanishing ball
self-driving|teenager|level fivetalk #15–17; interview #147–148millions of hours of data and still no level 5; a teenager learns in 20 hdata does not replace a world model: the argument against 'just scale' (07)
declarative knowledge|accumulation oftalk #30–31intelligence is not an accumulation of declarative knowledge; LLMs arethe generator accumulates; the map and the rudder are the part that 'does'
Piaget|when you don't knowtalk #25–27'intelligence is what you do when you don't know' (apocryphal)entropy gating: plan only where there is doubt (12)
Chomsky|Papert|perceptrontalk #21–22the Piaget–Chomsky debate; Papert defending the perceptron ten years after burying ithistorical context; no change
trillion|bytes|optic nervetalk #79–80; interview #158–16120–30 trillion tokens ≈ 10¹⁴ bytes; two million optic fibres at 1 byte/sthis project's 200 MB corpus is 10⁻⁶ of that: the scale is declared, not hidden (07)
four-year-old|16,000 hourstalk #81; interview #169four years of vision = 16,000 hours ≈ 10¹⁴ bytes, the same as all public textsame
perception module|currently perceivetalk #96, #133perception produces only the representation of what is perceived nowstate = perception + memory: the summed memory (11) and the bank (14)
action sequence|proposal for an actiontalk #119–122a sequence of actions is imagined and the one that minimizes the energy is searched for`mpc` mode: a sequence of H actions optimized in the latent (12)
zero if the task|distance to the tasktalk #137the objective gives zero when the task is accomplished and a distance when nottask head = 1 − p(z): zero at the goal (12)
kill anyone|deleterious|hurttalk #138guardrails ensure no state sequence 'kills or hurts anyone'the 'sem-numeros' guardrail is the toy version; the structure is the same (12, 13 §5)
10 millisecond|muscle|elementary actionstalk #148–149you cannot plan the trip in 10 ms muscle actionswhich is why level 2 plans in blocks, not tokens (09)
taxi|airport|elevatortalk #151–163; interview #266–267you don't know how long the taxi takes: planning in detail is impossible for lack of informationcompass.subgoal_until + subgoal_ruler: persistent subgoal with arrival (13 §11: token ruler never arrives; block ruler ties with replanning — a block of 8 is too short to persist)
PhD|unsolved|nobody knowstalk #172–173hierarchical planning is 'an unsolved problem; a great PhD topic'`plano2` is an attempt; chapter 09 does not claim to have solved it
generative model|reconstructtalk #180, #22115 years trying world models, 10 failing for being generative; reconstructing Y 'in its most minute details'the map predicts state, not tokens; the token head stays only for the generator (05)
blurry|GAN|two framestalk #214predicting two frames gives blur: the average of everything that can happen`futures.py`: in text, given the token, there is no blur to correct (13 §8)
autoencoder|identity functiontalk #246, #296an unconstrained autoencoder learns the identity: collapsethe identity floor is the collapse measured in every table (06)
representation space|joint embeddingtalk #208, #218; interview #402, #428predict in representation space; JEPA is not an alternative to the Transformerthe whole architecture (02, 05, 10)
MCR|Barlow|VICReg|information maximizationtalk #253–260MCR, MCR², W-MSE, VICReg, Barlow Twins: six years of information maximizationthe lineage of 04; VICReg and SigReg implemented
differentiable|back-propagatetalk #265–266a differentiable measure of information is needed to backpropagatethe interpreter probe is differentiable and enters training: --world-enc2-probe (13 §9, round sd1)
dependency|not a function|relationtalk #279–280; interview #103there is no single Y per X: it is a dependency, not a functionTransformer.energia_dependencia: minimum over plausible actions; order violation +130–142% against +38% given the token (13 §11)
landscape|valley|lake|level curvestalk #285–286energy is a landscape: the data in the valley, altitude rising as you leavethe energy-per-candidate plot on the panel is that landscape in one dimension
probabilistic|likelihood|special casetalk #105, #183; interview #98probabilistic models are a special case of energy models; the LLM is BERT removing only the last word−log p enters the energy as one term (γ), not as the whole (12)
isotropic|independent|same variancetalk #318, #374isotropic Gaussian = independent variables, each maximally informativeSigReg-z and the measured Gaussian floor (05, 06)
marginal|cumulative|staircasetalk #384–390project onto one direction, compare the cumulative distribution (staircase) with the Gaussian's`sigreg()`: Epps-Pulley over 64 projections (04)
Cramer|theorem|rotationtalk #399if all 1-D projections are Gaussian, the joint is isotropic (Cramér-Wold)same
spiral|nonlinear transformationtalk #408theorem: if the data are a nonlinearly transformed Gaussian, SigReg recovers the Gaussian up to rotationwhat justifies measuring `sigreg_razao` as 'ratio to the floor'
BYOL|bootstrap|DeepMindtalk #420–421the EMA came from DeepMind, to stabilize variance in RL; BYOL--world-ema (13 §4); correct_world.py --target-ema (13 §11)
I-JEPA|MAE|faster to traintalk #422I-JEPA beats MAE and trains fastercontext for the distillation family
DINO|best encoder|engineeringtalk #423–425DINO: distillation with 'a lot of engineering tricks'; the best generic image encoder`--dino`, round `wm-sobre-dino` (04, 05)
pendulum|push-T|25 stepstalk #407 and neighboursa planner with DINO-WM solves push-T and the double pendulum in under 25 stepshorizon 3 in energia mode is the minimal version
16 frames|window oftalk #446V-JEPA looks at 16 frames and predicts the next 16256-token window and blocks of 8: the same sliding-window structure
ball|disappearstalk #451vanishing ball: the prediction error 'shoots through the roof'inverted continuation +32% on the map against −3% on the identity (05)
segmentation|features|action recognitiontalk #440, #460heads on the encoder: action recognition, segmentationprobes (`probes.py`): the text analogue
abandon|academiatalk #464–466'abandon generative, probabilistic, contrastive models, RL' — 'don't work on LLMs in academia'the project keeps the generator and adds the rest: a deliberate choice (00)
AMI|industrial process|physical AIinterview #4; talk #47AMI Labs: 'AI for the real world' — robotics, industrial processestheir domain is continuous; ours is discrete — §7 measures the difference
constraint|engineer|walltalk #487–491'how do I put the constraint "don't bump into the wall" into representation space?' — 'a small head'the guardrail head, and the lesson of training it where it is read (13 §5)
two samples|open a doortalk #491'is the door open?' — a task cost trained with 'two samples'costs.py --prototype: two literal samples (13 §11: does not separate; ten sentences and regression)
emotion|fear|elation|happyinterview #5, #43fear, excitement, elation: anticipations of outcome`energia_prevista`; `anticipation.py` measures whether it anticipates (13 §8)
consciousness|definition ofinterview #7, #51no definition or measure of consciousness; 'not the right question'out of scope; no change
retina|upside downinterview #205–208the retinal image is upside down and the question 'how do we see right side up?' makes no sensean analogy for ill-posed questions; no change
diverse|tribe|wiredinterview #218–220brains wired slightly differently; diversity is the tribe's strength--seed, --seeds 2,3, mean ± std in the report (13 §10)
cube|rotateinterview #247–249rotate a cube 90° in your head: reasoning in mental space, not in actions`mpc` mode (12)
GPS|circuit|permutationsinterview #235–238the shortest circuit through the cities: search in the space of permutationsthe rudder is search; the modes differ in where they search (12)
glasses|assistant|inferenceinterview #380–385; talk #92, #104glasses with an assistant; 'the cost is used for inference, not for learning'cost heads only at inference (12)
Stargate|billion|infrastructureinterview #159 and neighbours60–80 billion a year in infrastructure, almost all for inferenceinference by search costs more per token: measured per mode (12)
regret|unsupervised|Hintoninterview #474–494the regret: taking self-supervised learning seriously too late; Hinton and backpropagationcontext; no change
Doubts answered by LeCun, and applied
Closed later, by the queue

The reading of LeCun does not end: every measured failure goes back to the text (chapter 13), and every piece the text asks for becomes a round (chapter 14).