Token does not measure delivered work. And between two people, it does not measure the same thing

What the column says

Aline Sordili published today, in UOL's economics section, a column whose title is the whole thesis: the word token already has seven meanings, but none of them measures delivered work.

The seven she lists: the representation units the model manipulates during processing; API billing units, priced differently for input, output and cached context; promotional credits; public or private subsidies; corporate benefits distributed internally; loyalty programmes; and transferable instruments with broader economic functions.

From there she describes two behaviours that already have names: tokenmaxxing, deliberately maximising consumption as a sign of AI adoption, and tokenminning, the opposite move, cutting consumption to discipline spending. Her argument is that both are bad for the same reason. When a measure becomes a target, people optimise the measure — true for tokens as it was for pageviews, calls answered and lines of code.

I agree with the thesis. What I want to add is something that only shows up when you open the tokenizer: even in the first of those seven senses, the most technical one, a token already does not measure the same thing for two different people.

What I measured

I took a paragraph carrying the column's own argument and wrote it twice, English and Portuguese, saying the same thing. Then I ran both through GPT-2's tokenizer, the one the course takes apart.

The same content costs 1.89 times more in Portuguese. Word by word it gets sharper still: 'productivity' is one token, 'produtividade' is four; 'work' is one, 'trabalho' is three; 'delivered' is one, 'entregue' is three. Even 'não' costs two, and 'ação' costs three.

If the ruler is the token, two people doing the same work, at the same quality, in the same time, produce numbers that differ by almost a factor of two — and the only difference is the language each one writes in.

Nobody wrote more. Nobody delivered more. The number doubled because the sentence was in Portuguese.

Why it happens, and why it is not pricing

This is nobody's commercial decision, and no discount fixes it. It is the shape of the vocabulary.

GPT-2's tokenizer uses BPE: it starts from the 256 possible bytes and repeatedly merges the most frequent neighbouring pair in the corpus, 50,000 times, ending with a vocabulary of 50,257 entries. What gets stored is an ordered list of merges, merges.txt, and that list is a statistical portrait of the text it saw in training.

GPT-2's corpus was WebText: 8 million web pages collected from links upvoted on Reddit, overwhelmingly in English. So the merges useful for English were learned, and many of the ones for Portuguese were not. 'produtividade' is not a piece; it is four pieces reassembled on the spot.

The consequence is not only price: it is memory, time and context. A Portuguese text takes up almost twice the positions, and attention costs the square of the number of positions. Half the usable context, for the same text.

What this changes in her argument

The column says the honest comparison is with other infrastructure units: API calls, CPU or GPU hours, cloud credits — and that not even those are interchangeable, because each measures a different aspect of the service. That is exactly right.

What I add is that the token is not even a stable unit inside its own technical sense. A GPU hour is an hour for everyone. A token is not a token for everyone: it depends on the language, and on which tokenizer the model uses. GPT-1 has 40,478 entries, GPT-2 has 50,257, RoBERTa has 50,265, and the IDs bear no relation to each other.

In other words: before you reach the good criticism — that consumption is not delivery — there is an earlier problem, that consumption is not even comparable consumption.

This does not weaken her thesis. It strengthens it. If the metric is already unstable at the most technical and best-defined of the seven senses, using it as a productivity indicator on a corporate dashboard is even less defensible than it looked.

The personal part

I write a course that takes GPT-2 apart line by line, and a reader pointed out a flaw in it, rightly: I was using token, layer, model and mask as if artificial intelligence owned those words. It does not. Token was already the unit a compiler separates when reading source code, the credential a server issues after login, and the asset registered in a blockchain contract. None of those meanings stopped being valid.

Because of that, the course gained a glossary: 36 terms, each with its machine-learning sense, the senses the same word carries in the rest of IT, and the chapters where it appears.

Aline's column arrives at the same place by another road — through economics, not engineering — and brings three senses I did not have: promotional credit, subsidy and corporate benefit. Noted.

That is why these two pieces are worth reading together. Whoever reads the word from inside the model and whoever reads it from inside the budget are finding the same problem: the word grew too big to be a number.