GPT-2 · RoBERTa Lab · Part III — The BERT family
12

The detector:
fine-tuning and its limits

How a language model is turned into a classifier — and why this particular tool must not be used to decide anything.

Chapter 12 of 15 Prerequisite: chapters 10 and 11

1 What this model is

When OpenAI released GPT-2, in 2019, there was real concern about misuse — generating disinformation at scale. The response was to release, alongside it, a detector: a RoBERTa fine-tuned to classify text as "written by a human" or "generated by GPT-2".

The gesture is elegant and worth appreciating: OpenAI built the hunter with the architecture opposite to that of the prey. A decoder generates; an encoder judges. It is the natural close of the course.

The anatomy, measured

roberta-base-openai-detector

  total parameters          124,647,170
  RoBERTa body              124,055,040   (99.53%)
  classification head           592,130   ( 0.47%)

  labels: {0: 'Fake', 1: 'Real'}

Look at the proportion. 99.5% of the model is the pre-trained RoBERTa, reused whole. The task-specific part — the one that knows how to tell human from machine — is half a percent.

2 Fine-tuning: how it is done

This is the mechanism that dominates the practical use of pre-trained models, and it has three steps:

  1. Rip off the old head. RoBERTa was pre-trained with an MLM head, which projects 768 dimensions to 50,265 (one score per vocabulary token). For classification, it is of no use. Discard it.
  2. Bolt on a new head. A tiny network that projects 768 → 768 → 2. Randomly initialized.
  3. Train everything together, with labeled data and a low learning rate (typically 2×10⁻⁵, versus 10⁻⁴ for pre-training). The low rate is essential: you want to adjust the existing knowledge, not destroy it.
Why this works so well

The RoBERTa body already knows what language is: grammar, semantics, style, register. That was the expensive part — 160 GB of text and thousands of GPU hours.

The task "human or machine?" does not require learning language again. It only requires learning to read a specific signal in a representation that is already rich. That is why a few thousand labeled examples and a few hours of training suffice, instead of months.

It is the same principle as hiring someone who already knows how to read and training them as a proofreader: you teach the reviewing criteria, not the alphabet.

Where the head reads — and why the <s>

The head needs one vector to represent the whole text, but the encoder returns one vector per token. Which one to use?

The BERT/RoBERTa convention: the vector of the first token, the <s> (called [CLS] in BERT). And there is a reason; it is not arbitrary:

Note that this only works in an encoder. In a causal decoder, the first token sees only itself — it is the most ignorant token in the sequence, not the best informed. That is why decoder-based classifiers use the last token, not the first.

logits = W2 · tanh(W1 · h<s>)  ∈  ℝ² 768→768 with tanh, then 768→2. Softmax over the two logits gives P(Fake) and P(Real). These are the 592,130 parameters in the table above.
Two real gotchas in these files

1. The roberta-large-openai-detector does not ship the labels. Its config.json shows LABEL_0/LABEL_1 instead of Fake/Real. The order is the same as in the base version (0 = Fake, 1 = Real), but if you blindly trust id2label you will report inverted results without noticing. The lab/common.py module fixes this explicitly.

2. There is an orphan pooler in the weights. On loading, transformers warns that roberta.pooler.dense.weight is "UNEXPECTED". It is a leftover from pre-training: RobertaForSequenceClassification reads the <s> directly, without going through the pooler. The warning is harmless — but it is worth knowing how to read warnings like this instead of ignoring them by reflex.

3 Does it work? Yes — inside its own world

Text generated by gpt2-large, judged by roberta-base-openai-detector. Measured on this machine.
Generation strategyP(human)Verdict
greedy0.0002MACHINE
T=0.7, top_p=0.90.0002MACHINE
T=1.0, top_p=0.950.1098MACHINE
T=1.40.0002MACHINE

Practically perfect. The detector was built for this exact task and performs it very well.

Methodological note: the expectation was to find here a pattern of the kind "the higher the temperature, the harder to detect". It did not appear — T=1.4 is caught just as easily as greedy. The absence of the pattern is recorded, instead of an explanation for an effect the numbers do not show.

4 Where it breaks — and the two failure modes

Failure 1: false positives on formal human text

All written by humans. The last two rows are errors.
Human textP(human)Verdict
Melville, Moby Dick (1851)0.952human
Scientific abstract0.975human
Informal technical account (bug debugging)0.186MACHINE ✗
Instruction manual0.436machine? ✗
Regulatory text0.034MACHINE ✗
Why the false positives are the real harm

A false negative (machine text passing as human) is a nuisance. A false positive — human text accused of being machine-made — is an accusation against a person.

Detectors of this kind were widely used by teachers to accuse students of cheating. In that context, a false positive is not a statistic: it is a student failed unjustly, unable to prove their innocence.

And notice who writes in a more formal, standardized and predictable way: those writing in a second language, those who learned to write in a rigid academic context, those who use an automatic spell checker. The false positives are not uniformly distributed — they fall disproportionately on those who are already at a disadvantage. There is literature documenting exactly this for non-native writers of English.

Failure 2: modern models go unnoticed

I gave both detectors a text written by an LLM from 2026:

DetectorP(human)Verdict
roberta-base-openai-detector0.9940HUMAN ✗
roberta-large-openai-detector0.9993HUMAN ✗

Complete failure, and with very high confidence — which is worse than failing with uncertainty. The model has no way of knowing it is outside its domain.

The machine-learning lesson embedded here

A classifier learns the boundary between the distributions it has seen. This one saw human text from 2019 and GPT-2 text from 2019. Text from a 2026 model looks like neither — it is out of distribution, and the model has no mechanism whatsoever to recognize that.

Worse: classification models are typically poorly calibrated out of domain. They do not become uncertain — they become confidently wrong, as you see in the 0.9993 above. This is one of the most important pitfalls of applied ML, and you have just seen it live.

Failure 3: it is an arms race lost by construction

Detecting machine text is adversarial: every new model makes the detector more obsolete. And the target moves faster than the detector can be retrained.

OpenAI itself acknowledged this in practice. It released its own, more modern detector in January 2023, and shut it down in July of the same year — six months later — citing a low accuracy rate. When the maker of the generators gives up on detecting them, that is a strong signal.

5 The answer to Exercise 0

Exercise 0 in chapter 00 asked for an estimate, without looking it up, of how accurate this detector would be in 2026 — and warned that the answer was uncomfortable. Here it is:

Verdict

Do not use this model to decide anything about a person. Against GPT-2 it works very well. Against anything from 2023 onward, it is worse than useless — it is confidently wrong, which gives a false impression of rigor.

As an object of study, however, it is excellent, and that is why it is in this course: it demonstrates fine-tuning, the use of the <s>, the economics of reusing 99.5% of a pre-trained model, and — perhaps most valuable — the difference between working on paper and working in the world.

6 In the lab

.venv\Scripts\python.exe lab\09_detector.py

The script reproduces everything above on your machine: the model's anatomy, the full cycle (GPT-2 generates → RoBERTa judges), the base vs large comparison, and the two failures. Run it to the end — section 5 is the point.

Exercise 12

  1. Write a paragraph yourself, in Portuguese and in English, and run it through both detectors. Do the results agree? What happens if you write in a deliberately formal way?
  2. The base detector gave 0.186 and the large gave 0.976 for the same human technical text. If you had to build a system, how would you use that disagreement to your advantage?
  3. The classification head has 592,130 parameters. Check the arithmetic: 768×768 + 768 (first layer) + 768×2 + 2 (second). Does it add up?
  4. Hard: propose a way to make a detector of this kind more robust to future models. Then argue why your proposal will also fail. (If you can do both, you have understood the problem.)
References

The primary sources for the concepts in this chapter. The numbers in the text were measured in the lab; the ideas come from here.

Technical terms here carry their machine-learning meaning. The course glossary shows that meaning next to the others the same words have in IT.

  1. Solaiman, I. et al. (2019). Release Strategies and the Social Impacts of Language Models. Technical report, OpenAI. arXiv:1908.09203
  2. Liu, Y. et al. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. Preprint. arXiv:1907.11692
  3. Howard, J. and Ruder, S. (2018). Universal Language Model Fine-tuning for Text Classification. ACL 2018. arXiv:1801.06146

End of Part III. You have walked both branches of the tree in full.
Part IV leaves theory behind: OpenAI's original code, and how to actually serve these models.