Rafhael Oliveira · IA
Experiment · 21 chapters · updated on 12 Sept 2026

Antivision · Latent Rehearsal

What changes when a language model predicts in latent space and decides by energy — an ELLM — and how that goes from theory to practice, with measured results, including the negative ones.

Not a course. An experiment of proof and continuous improvement: every piece is measured against a control, what fails is recorded with the same prominence as what works, and the next run starts from what the previous one measured. The chapters are the diary of that experiment, in the order it happened.

For whom already understands the Transformer and wants to follow a real experiment: a world model that rehearses in latent space before emitting the token, measured run by run, with the failures on record.

What you walk away knowing: what JEPA really is, why collapse happens, how energy picks the token, and what worked and what did not, with numbers.

The ideas are LeCun's. The conversion to text, and the risk, are ours.
Start with chapter 00Open chapter 00All PDFs by e-mail
Experiment in progress · updated on 12 Sept 2026

Experiment status

Read straight from each run's training history. No number typed by hand.

60runs measured
41complete duels
3improved the generator
35cost the generator
0results in the queue
21chapters

In the queue

Cells that still say 'awaiting' in the chapters. Each becomes a number when the run closes.

    What has closed

    The six best duel runs against the control. The full table is further down.

    RunValidationvs control
    reg1-lat1ocosine + DEEP compass5.2233-0.65%
    reg1-wm1emav3 with EMA teacher target, I-JEPA (correction)5.2405-0.33%
    reg1-bar0.5-lat1odeep compass + Barlow Twins5.2442-0.26%
    dueloRoPE + RMSNorm + SwiGLU, no regulariser5.2616+0.07%
    reg1-wm1detv3 control: stop-grad, no SigReg-z5.2653+0.14%
    reg1-lat1o-s2seed 2cosine + DEEP compass5.2867+0.55%

    See the runs, one by one →

    Chapter diary

    One chapter per piece, in the order the experiment happened. The badge says whether the run that closes the chapter has run.

    1. 00
      Antivision: an ELLM, a language model that rehearses in latent space before speakingAn LLM picks the next token and finds out afterwards where it ended up. We built one that imagines first: an ELLM.highlightclosedupdated 12 Sept
    2. 01
      The custom tokenizer: why 16 thousand tokens suit this corpus better than 50 thousand, and what the whole table showsA 7-million-parameter model used fewer bits than the 124-million GPT-2 in Portuguese. Not because it is better.closedupdated 12 Sept
    3. 02
      Two architectures, one file: the fair duel between 2019 and what came afterA new piece is presented under the conditions of whoever built it. We redid the comparison under ours: same data, same seed, same budget.closedupdated 12 Sept
    4. 03
      The geometry of the embedding: training starts with a perfect space and ruins itWe tried semantic search with RoBERTa. Every similarity came out above 0.99. Sentences on opposite subjects looked identical.closedupdated 12 Sept
    5. 04
      The lineage of collapse: five answers from computer vision, each measured on textA system trained to predict in latent space has a trivial solution: ignore the input and always return the same vector. Perfect and useless prediction.closedupdated 12 Sept
    6. 05
      The rudder and the world model: three versions of the same question, each measured against the lastThe question is a single one: does the model know where it is going before it goes?highlightclosedupdated 12 Sept
    7. 06
      Method and mistakes: the rules of a legitimate comparison and the catalogue of our own errorsHalf of the best decisions in this project came from finding out we were wrong.closedupdated 12 Sept
    8. 07
      Scale and budget: what it costs to reach GPT-3, and why VRAM is almost never the limitReaching the scale of GPT-3 Small costs less than eight hours of a single GPU. What is missing is not the card. It is text.closedupdated 12 Sept
    9. 08
      The workbench: serve, x-ray and train from the browserWe measured where the time of a request goes. 93.3% is serial generation, token by token.closedupdated 12 Sept
    10. 09
      The hierarchical architecture: the whole project aligned to the four patterns of the world modelLeCun has a slide with four lines: abandon X in favour of Y. We audited our code against each one.closedupdated 12 Sept
    11. 10
      What JEPA is, according to LeCun: two talks read concept by concept, and where each one appliesThe video that started it all: Yann LeCun explaining what LLMs still lack to understand the world. We read it concept by concept and applied it to text.closedupdated 12 Sept
    12. 11
      Memory and horizon: two claims from LeCun that became experiments, and both hold for textLeCun says: each level of abstraction predicts further than the one below. We measured. It holds for text.closedupdated 12 Sept
    13. 12
      Learned costs and inference modes: the three searches and the two sources of energy, measuredOn the first reading, no inference mode governed. Free generation had more heading than any planner.highlightclosedupdated 12 Sept
    14. 13
      The corrections from LeCun's talks: failure, passage, flag, measurementA queue of 22 runs left four measured failures. We took each back to LeCun's text with a single question.highlightclosedupdated 12 Sept
    15. 14
      Consulted memory: a bank of abstract states read by attentionSummed memory persists, but dilutes: the whole past is worth less than each half alone.closedupdated 12 Sept
    16. 15
      What it costs and what it delivers: us, GPT-1 and GPT-2 on the only ruler valid across tokenizers"Our model has perplexity 31, GPT-2 has 29, so GPT-2 is better." That means nothing.closedupdated 12 Sept
    17. 16
      The minimum was off the list: the day I measured my own search by energyFor weeks I said my system picked the next word by "searching for the lowest energy". I went and measured it. In 96 tests, it never found the lowest energy once.highlightclosedupdated 12 Sept
    18. 17
      ELLM: the right name for a language model that predicts in latent space and decides by energyI called my model an LLM for months. The name was wrong — and not because of its size.highlightclosed
    19. 18
      Searching better does not fix the energy: three searches, the same textI fixed my model's search so it finds the true lowest energy, among 16,392 options. The text did not get better.recenthighlightclosed
    20. 19
      The inversion that did not repeat: the world model's cost appears once the ruler stops swingingI wrote in a chapter that my world model made the generator 3.38% better. With more seeds and a fixed ruler, it costs.recenthighlightclosed
    21. 20
      What changed on 17/09: two new chapters and two correctionsIn one day the experiment gained two chapters and corrected two claims I had already published.recentclosed
    PDF

    The whole collection as PDFs, in your inbox

    All 21 chapters, in Portuguese, English or both. Leave your e-mail and the links arrive in one send. You join the list and get new pieces as they come out — to leave, it is one click at the bottom of any e-mail.

    PDF version
    Experiment in progress

    The duel runs

    Same data, same 700 steps, same seed. Validation of the best checkpoint; the difference is against the control.

    RunWhat it testsValidationvs controlMap gainChapter
    reg1-lat1ocosine + DEEP compass5.2233-0.65%—0405091112131418
    reg1-wm1emav3 with EMA teacher target, I-JEPA (correction)5.2405-0.33%+0.761013
    reg1-s2seed 2+ cosine regulariser (the CONTROL)5.2409-0.32%—
    reg1-bar0.5-lat1odeep compass + Barlow Twins5.2442-0.26%—
    reg1control+ cosine regulariser (the CONTROL)5.25770.00%—000204050609101315161819
    dueloRoPE + RMSNorm + SwiGLU, no regulariser5.2616+0.07%—0006091618
    reg1-wm1detv3 control: stop-grad, no SigReg-z5.2653+0.14%+0.84040509121316
    reg1-lat1o-s2seed 2cosine + DEEP compass5.2867+0.55%—
    reg1-lat1o-s3seed 3cosine + DEEP compass5.2924+0.66%—
    reg1-wm1n2eema-s2seed 2hierarchy + level-2 encoder + EMA target (correction)5.2976+0.76%+0.83
    reg1-wm1n2eemahierarchy + level-2 encoder + EMA target (correction)5.3221+1.22%+0.80
    reg1-s3seed 3+ cosine regulariser (the CONTROL)5.3246+1.27%—

    Outside the duel

    Runs on the full corpus, with a bigger budget. Not comparable with the duel control.

    RunWhat it testsValidationMap gainChapter
    reg1-s2-escala3.5762—
    reg1-s3-escala3.6091—
    reg1-s3-escalafixseed 33.6136—
    reg1-wm1n2eema-escalahierarchy + level-2 encoder + EMA target, on the full corpus3.6236+1.64
    reg1-s2-escalafixseed 23.6237—
    reg1-wm1n2eema-s2-escala3.6401+1.62
    reg1-escalafixthe control on the full corpus, with fixed validation batches (chapter 19)3.6729—19
    reg1-wm1n2eema-escalafixhierarchy + EMA target on the full corpus, fixed validation: +1.77% across three seeds3.6841+1.68
    reg1-wm1n2eema-s3-escalafixseed 33.7051+1.65
    reg1-wm1n2eema-s2-escalafixseed 23.7141+1.65
    reg1-escalathe control trained on the full corpus (1,960 steps)3.7503—0002030405060709101112131415161719
    reg1-wm1n2eema-s3-escala3.7974+1.60

    status read from the runs on 17 Sept 2026.