Rafhael Oliveira · IA

The detector: how a language model becomes a classifier, and why this one should decide nothing

Fine-tuning in practice, the reuse of 99.5% of a pre-trained model, and the two failure modes.

1 min read· chapter: 8 min

Landed here from LinkedIn? This is chapter 12 of the course GPT-1 and GPT-2, taken apart. It reads on its own; if you want the beginning, it is chapter 00.

OpenAI built the hunter with the architecture opposite to the prey's. And since 2023 it is worse than useless. When GPT-2 came out in 2019, there was real concern about disinformation at scale. The answer was to ship a detector alongside it: a RoBERTa fine-tuned to say whether a text was written by a human or generated by GPT-2. A decoder generates; an encoder judges. It is elegant. Against GPT-2 it works very well. Against anything from 2023 onward, it is confidently wrong, which gives a false impression of rigour. As an object of study, it is excellent. As a tool to decide anything about a person, it should never be used.

The three points

  1. Fine-tuning: a new classification head over the start token, and the whole model adjusted with little data.
  2. It works inside its world: GPT-2 text from 2019. Outside it, the two failure modes appear.
  3. The verdict is explicit: do not use it to decide anything about a person. Use it to understand fine-tuning.

From the chapter, the beginning

How a language model is turned into a classifier — and why this particular tool must not be used to decide anything.

When OpenAI released GPT-2, in 2019, there was real concern about misuse — generating disinformation at scale. The response was to release, alongside it, a detector: a RoBERTa fine-tuned to classify text as "written by a human" or "generated by GPT-2".

The gesture is elegant and worth appreciating: OpenAI built the hunter with the architecture opposite to that of the prey. A decoder generates; an encoder judges. It is the natural close of the course.

Look at the proportion. 99.5% of the model is the pre-trained RoBERTa, reused whole. The task-specific part — the one that knows how to tell human from machine — is half a percent.

Keep reading chapter 12 →Ler em portuguêsThe PDFs by e-mail →
What the chapter covers · 6 sections · 8 min
  1. What this model is
  2. Fine-tuning: how it is done
  3. Does it work? Yes — inside its own world
  4. Where it breaks — and the two failure modes
  5. The answer to Exercise 0
  6. In the lab
All chapters of the course · GPT-1 and GPT-2, taken apart
  1. 00 · The seven fossils: why GPT-1, GPT-2 and RoBERTa are a historical cross-section, not a list
  2. Glossary · Course glossary: what token, layer and model mean in AI, and in the rest of IT
  3. 01 · Why GPT-2 sees neither letters nor words, and what that costs in Portuguese
  4. 02 · Embeddings: from a meaningless integer to a vector that knows king and queen are related
  5. 03 · Attention: the single operation that replaced twenty years of recurrent networks
  6. 04 · Twelve heads instead of one, and the single line of code that separates GPT from RoBERTa
  7. 05 · Why GPT-2 needs 48 floors to answer, and what happens inside each one
  8. 06 · What scale really buys: from 124 million to 1.5 billion, measured
  9. 07 · Temperature, top-k and top-p: the decision that changes the text more than switching models
  10. 08 · Perplexity: the one number that says what the model learned, without anecdotes
  11. 09 · GPT-1 to GPT-2: same size, half the perplexity. Four decisions explain it
  12. 10 · RoBERTa: when the advance is training better what already existed
  13. 11 · Encoder vs decoder: one line of code and every consequence that descends from it
  14. 12 · The detector: how a language model becomes a classifier, and why this one should decide nothing
  15. 13 · Reading GPT-2's original code: 174 lines against everything you learned
  16. 14 · Serving models: from function call to HTTP server, and what production does on top
  17. 15 · The lab guide: every command, every script, and everything that was measured
  18. 16 · The ceiling of this path: LeCun's argument, read from inside GPT-2

After this: the Antivision experiment

Rafhael Oliveira
Rafhael Oliveira

Writes the GPT-2 course and runs the Antivision experiment, measuring every run before publishing the number. The LinkedIn posts start here and end in the chapters.

Follow on LinkedIn →