Learned costs and inference modes: the three searches and the two sources of energy, measured

On the first reading, no inference mode governed. Free generation had more heading than any planner. Energy is the sum of two things: what the system wants and what it cannot do. Inference is the search that minimises it. Three searches: energy, gradient, predictive control. With weight 1, all came in below free generation. The map predicted the next state with cosine 0.93, but the energies of the eight candidates were almost equal. The cause was none of the ones we suspected. With weight 0.1, a learned head and cost on the real state, the rudder governed: more heading and more plausible at the same time. The only mode that improves both axes. All three pieces were needed together.

The three points

1. The energy: task (distance to the heading) plus guardrail (minus log p). The task can be a heading in text or a head learned from three positive sentences.
2. The independent judge: a head trained elsewhere says whether the generated text has the requested heading. Without it, the planner judges itself.
3. The best configuration: learned head, cost on the real state, weight 0.1. Affinity 0.355 against 0.256 for free, with perplexity 36.7 against 48.5.

From the chapter, the beginning

The energy is the sum of two things — what the system wants and what it must not do — and inference is the search that minimizes it. This chapter measures the three searches and the two sources of energy on a trained world model.

E(z) = task(z) + γc Σ guardraili(z) + γ · (−log p) accumulated along the trajectory