The seven fossils: why GPT-1, GPT-2 and RoBERTa are a historical cross-section, not a list

Seven files on Hugging Face record the exact moment language AI split in two. Between 2018 and 2019 the whole field forked into two branches. On one side, the branch that generates text one word at a time, looking only backward: GPT-1, GPT-2 small, medium, large, XL. That lineage would lead to ChatGPT. On the other, the branch that reads the whole sentence and generates nothing: BERT, RoBERTa. OpenAI's seven models on Hugging Face are not a list. They are the fossils of that moment. And the difference between the two branches fits in one line of code.

The three points

1. The seven models are two branches: decoder (generates) and encoder (understands). Everything that came later descends from one of them.
2. The track goes from text becoming numbers to the model becoming a server, and every chapter has a lab that runs on your machine.
3. No number in the course is quoted from a paper. All were measured on an ordinary machine, and the guide says how to repeat them.

From the chapter, the beginning

What you will learn, in what order, why in that order — and where each piece lives on your machine.

This course takes apart GPT-2 and RoBERTa — the seven models the openai-community organization published on Hugging Face. They are not a random list: they are a historical cross-section. Between 2018 and 2019, the entire field of language processing split into two branches, and these seven files are exactly the record of that moment.

On one side, the decoder branch: GPT-1 → GPT-2 small → medium → large → XL. This is the lineage that generates text, one word at a time, looking only backward. It would lead to GPT-3, to ChatGPT and to everything that came after.

On the other side, the encoder branch: BERT → RoBERTa. This is the lineage that understands text, looking at the whole input at once, forward and backward. It remains widely used for classification, search and information extraction.