Searching better
does not fix the energy

1 The question chapter 16 left

In energy mode, the generator picks each token in two steps. First it builds a short list of candidates. Then it imagines, in latent space, where each candidate would take the text, and gives each one an energy with two terms: how far the predicted state lies from the requested heading, and how improbable the token is according to the token head itself.

E(a) = 1 − cos(Pred(z_t, e_a), heading) + γ · (−log p(a))

Chapter 16 showed that the short list — the 8 most probable tokens — often does not contain the lowest-energy token: at γ = 0 this happens in all 96 positions measured. The response was a new mode, todos (all), which computes the energy of each of the 16,392 tokens in the vocabulary and takes the lowest. One question remained, the first on that chapter's list of open questions: does finding the true minimum improve the text, or only lower the energy?

2 A defect found along the way

The first measurement came out odd. At γ = 0.1, the todos mode gave a heading affinity of 0.780, well above anything chapter 12 had measured, but with perplexity 491; in energy mode without a cost head perplexity reached 2,098. The sweep in chapter 16 said that at γ = 0.1 the true minimum is already among the 8 most probable tokens in 85% of positions. A search that finds the minimum should not produce text so much worse than the short list.

The reason was in the code. The function that orders the vocabulary (_todos_por_energia, in server/ours.py) used only the heading term and left out the γ · (−log p) term. The eight candidates were therefore the minima of the pure heading — the syllable fragments that chapter 16 §5 showed no policy would pick — and γ only reordered that set. At γ = 0 both orderings coincide, which is why the measures in chapter 16, taken by model/inference.py with the whole energy, do not change.

The ordering now uses the whole energy, the same one inference.py measures. Every number below is after the fix.

3 Three places to search, the same trade-off

To see whether where one searches is what matters, we compared three search sets, always choosing by the same energy:

1. policy: the 8 most probable tokens (the usual behaviour);
2. whole vocabulary: all 16,392 tokens (the todos mode);
3. nucleus: only the tokens that, summed from the most probable down, cover a fraction of the probability — the same idea as nucleus sampling (top-p). The search is exact inside it. This is the new nucleo mode, with the mass in nucleus_mass.

The idea behind the nucleus was to keep a firm plausibility constraint, like the short list's, and search for the exact minimum inside it.

The middle columns, those of the best heading configuration (cost head applied to the real state), give the answer. The four bold rows land at practically the same point: affinity between 0.476 and 0.488, perplexity between 82 and 84. Three of them are the three searches at γ = 0.1; the fourth is the usual short list at γ = 0. Searching the nucleus or the whole vocabulary at γ = 0.1 takes the text exactly where lowering γ already took it.

Put another way: there is a trade-off between following the heading and writing plausible text. Lowering γ moves along that trade-off; searching better also moves along it, and does not shift it. None of the searches bought more heading for the same price in perplexity.

One small exception is worth recording: on the guardrail, the 50% nucleus zeroes the digits at perplexity 75.6, against 144.7 for the short list at γ = 0 and 247.2 for the whole vocabulary. It is a 3-seed measure on a single prompt, and it stands as a lead to repeat, not as a result.

4 Why the searches tie

The rows for the 80% nucleus and the whole vocabulary are not similar: they are identical, to the third decimal. That is not a defect; it is what two measurements already taken predict together.

The first is the γ sweep, taken with model/inference.py over 96 validation positions. It says how often the lowest-energy token is among the policy's k most probable:

The second is the size of the nucleus. The mini policy is flat: its probability spreads over many tokens.

Together they explain the tie. At γ = 0.1, the lowest-energy token is among the 128 most probable in every position. An 80% nucleus holds about 2,000 tokens and a 95% one almost 9,000; both contain the same eight lowest-energy tokens as the whole vocabulary. The searches differ; the candidates they return do not.

5 What holds plausibility is rank, not mass

The table in §3 therefore has a more precise reading than "the short list is better". What makes the short list produce plausible text is the cut by rank: no token below the eighth most probable gets in. The tokens the energy prefers at γ = 0.1 are almost always between rank 9 and rank 128 of the policy — inside any reasonable nucleus, but outside the short list — and that is where the perplexity of 83 comes from.

The γ · (−log p) term does not imitate that cut. A token at rank 50 has low probability, but not low enough for the term to outweigh the gain on the heading when γ is small. When γ is high the term weighs enough, and the energy goes back to being almost the policy itself (16 §4 showed this at γ = 1). In our energy, plausibility lives in a continuous term, and what worked was a cut.

6 What changed in the code

7 Limits of this measurement

• It is 4 prompts × 32 tokens and one generation seed in the affinity columns. The table supports the order of magnitude — 0.355 against about 0.48, perplexity 37 against about 83 — not differences in the third decimal.
• Everything was measured on a single model, the mini with an end-to-end world model (wm1-sz1). Its policy is flat; in a larger model with a more concentrated policy the nucleus shrinks, and the tie in §4 may not repeat.
• There is one heading ("futebol gol campeonato"), and affinity comes from a cost head trained on a few sentences (chapter 12).

Measured on CPU on 17/09/2026. Data in out/cpu_1709/.