The workbench: serve, x-ray and train from the browser
A page with no framework over a FastAPI API, and the two measurements that decided the server.
Landed here from LinkedIn? This is chapter 08 of the experiment Antivision · Latent Rehearsal. It reads on its own; if you want the beginning, it is chapter 00.
We measured where the time of a request goes. 93.3% is serial generation, token by token. A local page, no framework, no build, no CDN. Works offline. Five tabs over the same API: generate, x-ray, train, compare, measure. Two measurements decided how the server was written. The first is that one: almost all the time is the serial generation loop, and that is why each extra token costs so much. The second is about a cache with a budget, and about what happens when it has none.
The three points
- Where the time goes: serial generation dominates. Tokenising, loading and responding are noise next to it.
- The cache with a budget: big models stay in memory up to a limit, and the limit is measured, not guessed.
- The Lab tab runs the project's scripts with a live log. The Models tab lists every training with saved weights, with curve and metrics.
From the chapter, the beginning
Serving, X-raying and training from the browser — and the two measurements that decided how the server was written.
The Playground selector has two groups: OpenAI (only gpt2-xl) and Ours (all checkpoints). The other four HuggingFace models remain downloaded and keep serving the labs — they just left the selector, so it does not become a list of things nobody picks.
What the chapter covers · 6 sections · 6 min
- What the bench is
- Where the time of a request goes
- The cache with a budget
- The Laboratory tab
- The Models tab
- The API
All chapters of the experiment · Antivision · Latent Rehearsal
- 00 · Antivision: an ELLM, a language model that rehearses in latent space before speaking
- 01 · The custom tokenizer: why 16 thousand tokens suit this corpus better than 50 thousand, and what the whole table shows
- 02 · Two architectures, one file: the fair duel between 2019 and what came after
- 03 · The geometry of the embedding: training starts with a perfect space and ruins it
- 04 · The lineage of collapse: five answers from computer vision, each measured on text
- 05 · The rudder and the world model: three versions of the same question, each measured against the last
- 06 · Method and mistakes: the rules of a legitimate comparison and the catalogue of our own errors
- 07 · Scale and budget: what it costs to reach GPT-3, and why VRAM is almost never the limit
- 08 · The workbench: serve, x-ray and train from the browser
- 09 · The hierarchical architecture: the whole project aligned to the four patterns of the world model
- 10 · What JEPA is, according to LeCun: two talks read concept by concept, and where each one applies
- 11 · Memory and horizon: two claims from LeCun that became experiments, and both hold for text
- 12 · Learned costs and inference modes: the three searches and the two sources of energy, measured
- 13 · The corrections from LeCun's talks: failure, passage, flag, measurement
- 14 · Consulted memory: a bank of abstract states read by attention
- 15 · What it costs and what it delivers: us, GPT-1 and GPT-2 on the only ruler valid across tokenizers
- 16 · The minimum was off the list: the day I measured my own search by energy
- 17 · ELLM: the right name for a language model that predicts in latent space and decides by energy
- 18 · Searching better does not fix the energy: three searches, the same text
- 19 · The inversion that did not repeat: the world model's cost appears once the ruler stops swinging
- 20 · What changed on 17/09: two new chapters and two corrections
Writes the GPT-2 course and runs the Antivision experiment, measuring every run before publishing the number. The LinkedIn posts start here and end in the chapters.
Follow on LinkedIn →