The minimum was off the list: the day I measured my own search by energy

For weeks I said my system picked the next word by "searching for the lowest energy". I went and measured it. In 96 tests, it never found the lowest energy once. The method was this: take the 8 most likely candidates and reorder them by energy. It looks like a search. Except the 8 candidates come from the very network I was trying to correct — if the true minimum sits outside that list, the search can never find it. I computed the exact minimum over all 16,392 options in the vocabulary, one by one, and compared. The result has two ends and both are uncomfortable. At the default setting, the list hits the minimum 100% of the time — but only because the energy had become, almost entirely, the network's own probability. When I give the objective a voice, the list misses the minimum 100% of the time. Either there is no rudder, or the rudder was never on the list.

The three points

1. The search by energy was not searching. With the network's 8 candidates, the true minimum was left out in 70% of positions on the world model's energy, and in 100% of them once the objective had weight. Not even 128 candidates push coverage past half.
2. There is no such thing as THE answer, and now there is a number: 12.9 different actions land within 5% of the minimum. Picking one is sampling, not inference — exactly what the talk means by calling this a relation rather than a function.
3. The model with the BEST validation loss is the one with the shallowest energy landscape: the energy of an arbitrary point in space comes out about four times cheaper there than in the regularized model, across all 18 repeated measures. The three collapse measures I logged at every evaluation gave no warning at all.

From the chapter, the beginning

The lecture makes two claims about energy that can be checked with a ruler: that inference is a search for the minimum, and that training an energy-based model means keeping the landscape from going flat. We measured both on our model. The first one caught us where it hurts: there is a setting in which what we called a search by energy did not find the minimum in a single one of the 96 positions tested — and another in which it always finds it, but only because the energy had become the policy itself.

From the energy-based models part of the lecture, two claims are checkable and had never been checked here.

The first is about inference. "Inference by optimization, not by forward propagation." Given an x, the compatible y are the ones that minimize the energy; finding them is a search, not a forward pass. And he insists on what that implies: "there's no single y for a given x — it's a relation, not a function", and therefore "you can give me multiple answers".

The second is about training. "To train an energy-based model you have to prevent collapse. The collapse problem would be manifested by the energy function being flat everywhere." And the two ways out: the contrastive one, "you generate points outside the region of data and you push the energy up"; and the regularized one, which he prefers, "minimizing the volume of space that can take low energy".