Lab
guide

1 Start here

# open PowerShell in the repository root and run, in order:

.venv\Scripts\python.exe lab\01_tokenizer.py
.venv\Scripts\python.exe lab\02_embeddings.py
.venv\Scripts\python.exe lab\03_attention.py
.venv\Scripts\python.exe lab\04_multihead_mask.py
.venv\Scripts\python.exe lab\05_block.py
.venv\Scripts\python.exe lab\06_scale.py
.venv\Scripts\python.exe lab\07_decoding.py
.venv\Scripts\python.exe lab\08_perplexity.py
.venv\Scripts\python.exe lab\09_detector.py
.venv\Scripts\python.exe lab\10_original_code.py

# lab 11 needs the server. In ANOTHER terminal:
.venv\Scripts\python.exe server\app.py
# and then:
.venv\Scripts\python.exe lab\11_api.py

Labs 06 and 08 load gpt2-xl (6 GB of RAM, ~3 min). If you are in a hurry or short on memory, append rapido:

.venv\Scripts\python.exe lab\06_scale.py rapido

2 Index: chapter → lab

3 File layout

GPT-2\
├── .venv\                      Python 3.14 + torch 2.14 (CPU) + transformers 5
├── models\hf\                  the 7 models, 13.2 GB in safetensors
├── repos\
│   ├── gpt-2\                  original TensorFlow code (for READING)
│   ├── GPT-3-Encoder\          JS tokenizer + vocab.bpe + encoder.json
│   └── gpt-3\                  175B samples and corpus statistics
├── lab\
│   ├── common.py                shared module — loads and configures everything
│   └── 01..11_*.py             the experiments
├── server\app.py             API compatible with TGI and TEI
├── tools\
│   ├── style.css               typography of the chapters
│   ├── generate_pdfs.py           HTML -> PDF (headless Chrome + page numbering)
│   ├── download_models.py       model download
│   └── verify.py            smoke test of the 7 models
├── src\conceitos_gpt-1-gpt-2\  HTML source of the 16 chapters (eng\ = English)
├── src\conceitos_antevisao\    the theory of the ELLM (Antivision experiment)
├── pdfs\gpt-1-gpt-2\          << the finished chapters; pdfs\antevisao\ those of the project
└── out\                        experiment outputs

Maintenance commands

# check that the 7 models load
.venv\Scripts\python.exe tools\verify.py

# regenerate all PDFs after editing an HTML
.venv\Scripts\python.exe tools\generate_pdfs.py

# regenerate just one
.venv\Scripts\python.exe tools\generate_pdfs.py 07

4 The model aliases

All scripts use short aliases, defined in lab/common.py:

# to write your own experiments
import common
tok, mod      = common.carregar_gerador("gpt2-medium")
tok, mod, rot = comum.carregar_detector("det-base")
tok, mod      = comum.carregar_base("gpt2", atencoes=True)

common.py points the cache to models\hf, turns on offline mode (nothing goes to the network), forces UTF-8 on the Windows console, disables gradients and fixes the missing labels of det-large.

5 Everything that was measured on this machine

6 Common problems

7 Where to go next

You are done. Four natural paths, in order of effort:

1. Fine-tune your own classifier. You saw that 99.5% of the detector is reused RoBERTa. Take a plain roberta-base, a labeled dataset of your own, and repeat the process from chapter 12. It is the most profitable practical skill in this course.
2. Implement GPT-2 from scratch in PyTorch and load the official weights into it. If the output matches that of transformers, you have truly understood. Andrej Karpathy's nanoGPT is a good reference for this step — and now you can read all of it.
3. Compare with a modern model. You already have qwen3:4b and qwen3:8b installed. Run the same perplexity and decoding tests on them and see what seven years changed — and what did not (the architecture is still the same).
4. Build a real RAG. Lab 11 showed why the detector is no good for embeddings. Download a model trained for that (BAAI/bge-small-en, intfloat/e5-small) and redo section 4 — this time working.

End of the course. The scripts are still there — change the prompts, break things, measure again.
That is how every number in these chapters came to be.