Antivision is an evolving experiment: every idea becomes a training run against the same control, the number goes into the chapter, and what fails is recorded next to what works. This page changes every time a run closes. The foundation, GPT-2 taken apart line by line, is in the course.
Yann LeCun, on world models
In this talk LeCun argues that predicting symbols is not enough for a world model, and proposes a complementary path: predict in latent space, choose by energy, avoid collapse by regularisation, and plan instead of react.
Antivision takes those ideas and does what he does not: brings them to text, and measures. Chapter 10 reads the talk concept by concept and says where each one entered the code.
Read the reading of the talk →In one day the experiment gained two chapters and corrected two claims I had already published.
Nine measurements closed, two new chapters, four corrected, no "pending" cell left open in the collection.A course and an experiment. The experiment begins where the course ends.
For whom already understands the Transformer and wants to follow a real experiment: a world model that rehearses in latent space before emitting the token, measured run by run, with the failures on record.
What you walk away knowing: what JEPA really is, why collapse happens, how energy picks the token, and what worked and what did not, with numbers.
For whom follows the subject through the news and wants to know what of it can be verified — and what cannot.
What you walk away knowing: what holds up or knocks down a claim going around, with the measurement made here and the method to repeat it.
For whom has never opened a model up, or uses an LLM every day and wants to know what happens between the question and the answer.
What you walk away knowing: how text becomes numbers, what attention does, what scale buys, and the single line of code that separates GPT from BERT.
Writes the GPT-2 course and runs the Antivision experiment, measuring every run before publishing the number. The LinkedIn posts start here and end in the chapters.
Follow on LinkedIn →