Why GPT-2 needs 48 floors to answer, and what happens inside each one

When you send ChatGPT a question, it climbs 96 floors before it becomes an answer. In GPT-2 it was 48. Every floor does the same thing. So why so many? Each floor has two rooms. In the first, the words talk to each other: that is where "it" figures out it refers to the trophy, not the suitcase. In the second, the model consults what it knows about the world: that is where knowledge lives, two thirds of everything it learned. But there is a detail that changes everything. If you stacked only the first room, a hundred floors would be worth exactly the same as one. The math does not allow more than that. What makes 48 floors worth more than one are four small pieces, and one of them is a plain sum. Without that sum, the model cannot even be trained.

The three points

1. Each layer has two rooms: attention, where the words look at each other, and the MLP, where knowledge lives. Two thirds of what the model knows sit in the second room, not the first.
2. Stacking attention alone gets you nowhere: a hundred layers are worth one. What changes that is a connection that adds the input back to the output, and it is what lets you train 48 layers without the training falling apart.
3. GPT-2 moved each layer's "volume knob", the LayerNorm, to before attention. It looks like a detail. It is one of the four decisions that doubled quality over GPT-1.

From the chapter, the beginning

LayerNorm, GELU, MLP and the residual connection — the four pieces that let you stack 48 layers without training blowing up.

Attention solves the transport of information: it moves content from one position to another. But notice what it is, mathematically: a weighted average of V vectors. That is, a linear combination.

And there lies the problem. Stacking linear operations gains nothing: the composition of two linear transformations is another linear transformation. A hundred layers of pure attention would have exactly the same expressiveness as a single layer. You would gain nothing by stacking.

We need non-linearity. And we need a place where the model can process the information attention brought in, instead of merely transporting it. That place is the MLP.