Encoder vs decoder:
the showdown

1 The line

We have reached the point the course had been preparing for since chapter 00. Here it is, in full, the architectural difference between GPT-2 and RoBERTa:

if causal:
    att = att.masked_fill(futuro, float("-inf"))

That's it. Everything else is the same or almost the same: the same Transformer block, the same attention equation, the same tokenization algorithm (byte-level BPE), the same GELU, the same LayerNorm, practically identical sizes (124 M vs 125 M, both with 12 layers, 768 dimensions and 12 heads).

gpt2 and roberta-base are structural twins that do opposite things. It is the perfect controlled experiment, and that is why studying them together teaches more than studying either one alone.

2 The experimental proof

This is not an argument — it is a measurement. In lab 04 you run two sentences that are identical up to the middle and different at the end, and measure how much the representation at each position changed:

A: "The cat sat on the mat and slept   peacefully all afternoon"
B: "The cat sat on the mat and screamed furiously  all afternoon"
                                 ^^^^^^^^^^^^^^^^ only this changes

GPT-2 (causal)                    RoBERTa (bidirectional)
 pos token      |diff|             pos token      |diff|
   0 'The'    0.00e+00               0 '<s>'     3.31e-01
   1 'cat'    0.00e+00               1 'The'     3.66e-01
   2 'sat'    0.00e+00               2 'cat'     2.88e-01
   3 'on'     0.00e+00               3 'sat'     3.91e-01
   4 'the'    0.00e+00               4 'on'      3.40e-01
   5 'mat'    0.00e+00               5 'the'     3.01e-01
   6 'and'    0.00e+00               6 'mat'     2.59e-01
   7 'slept'  7.86e+00  <- changes   7 'and'     3.36e-01
   8 ...      1.30e+01               8 'slept'   3.97e-01
                                     ...
                                    12 '</s>'    3.27e-01

ZERO up to the swapped word.       EVERYTHING changes, even <s>.
The past is untouchable.           One word rewrites the sentence.

It is not approximately zero. It is exactly 0.00e+00, bit for bit. The future does not exist for the past in a causal model. And in RoBERTa, swapping one word at the end alters even the opening token.

3 The tree of consequences

Everything that follows descends from that line. Follow the chain:

Branch A — the decoder

1. The mask prevents seeing the future.
2. → Therefore, predicting the next token is a legitimate task, with no leakage.
3. → Therefore, all n positions generate a training signal. Maximum efficiency.
4. → Therefore, the model learns P(next | previous), which is exactly what is needed to generate.
5. → Therefore, the past never changes when a token is appended.
6. → Therefore, the KV cache is possible, and generation drops from O(n²) to O(n) per token.
7. → Therefore, scaling is feasible, and scale brought in-context learning.

Branch B — the encoder

1. Without the mask, each token sees the whole sentence.
2. → Therefore, predicting the next token would be copying. Unviable objective.
3. → Therefore, another objective is needed: MLM, masking randomly drawn words.
4. → Therefore, only 15% of positions generate signal. 85% of the compute is wasted.
5. → Therefore, training is less efficient per token processed.
6. → But each representation is built with context from both sides, and is richer for understanding.
7. → And appending a token changes everything, so there is nothing stable to cache. Autoregressive generation is impossible.

4 The practical decision table

5 The third family: encoder-decoder

There is a third arrangement, worth knowing to complete the map. The original 2017 Transformer was like this, and T5 and BART follow the same design:

• A bidirectional encoder reads the entire input and produces rich representations.
• A causal decoder generates the output token by token, with one extra piece: cross-attention, where the queries come from the decoder and the keys and values come from the encoder.

It is the best of both worlds, at the cost of roughly twice the model. Ideal when input and output are distinct objects — translating from Portuguese to English, summarizing an article. Less useful for open dialogue, where the "input" and the "output" are the same continuous stream of conversation — which helps explain why the world of assistants converged on pure decoders.

6 The lineage, on one page

                    Transformer (2017)
                    encoder + decoder
                           |
              +------------+------------+
              |                         |
        DECODER only              ENCODER only
     (with causal mask)        (without causal mask)
              |                         |
         GPT-1 (2018)             BERT (2018)
         117M, BooksCorpus        MLM + NSP, WordPiece
              |                         |
         GPT-2 (2019)             RoBERTa (2019)
         124M-1.5B, WebText       dynamic MLM, no NSP,
         Pre-LN, byte BPE         GPT-2's byte BPE, 160GB
              |                         |
         GPT-3 (2020)                   +--> roberta-*-openai-detector
         175B, few-shot                      binary classifier
              |                             trained on GPT-2's outputs
         InstructGPT / ChatGPT                        ^
         fine-tuning + RLHF                           |
              |                                       |
         modern LLMs  -------- generate the text -----+
                                the detector judges

Note how the drawing closes: the two branches, separated in 2018 by one line of code, meet again in the detector. OpenAI built the hunter with the architecture opposite to the prey's. That is chapter 12.

7 In the lab

.venv\Scripts\python.exe lab\04_multihead_mask.py   (section 3 — the proof)
.venv\Scripts\python.exe lab\10_roberta_mlm.py         (the comparison)

Chapter 12 closes Part III with the most curious model in the collection: the detector.
And it brings an uncomfortable answer to question 3 of Exercise 0.