Search papers, labs, and topics across Lattice.
This paper conducts a compute-normalized comparison of five test-time scaling (TTS) methods across diverse open-ended generation tasks, revealing that while exploration strategies improve candidate quality, the exploitation phase鈥攚here final outputs are selected from these candidates鈥攅xhibits significant shortcomings. The authors find that state-of-the-art reward models correlate poorly with actual output quality, leading to near-random selection outcomes. Notably, while synthesis methods like Fusion show consistent improvements, they still only capture about 40% of the potential quality available from the candidate pool, indicating that the real bottleneck lies in the exploitation process.
Exploitation, not exploration, is the critical bottleneck in test-time scaling for language models, with selection processes yielding near-random results despite rich candidate pools.
Test-time scaling (TTS) improves language model outputs by spending additional inference compute - generating multiple candidates, searching over partial sequences, or iteratively refining drafts. These techniques yield large gains on mathematics and code, but have been developed and stress-tested almost exclusively on tasks where verification is straightforward. We conduct the first compute-normalised comparison of five TTS families across five open-ended generation benchmarks spanning medicine, law, finance, general chat, and creative writing - grounded in a unified framework that decomposes the effectiveness of each method's token budget into exploration and exploitation. The answer depends on which side of that decomposition you examine. Scaling exploration works: the best candidate in the pool improves steadily with compute across all settings. What breaks is exploitation - the step that converts a rich candidate pool into a final output. With state-of-the-art generators, reward models correlate at only $\rho_v \approx 0.12$ with true quality, rendering selection near-random regardless of budget. Tree search amplifies this failure through diversity collapse. Refinement helps on one of five benchmarks; its apparent gains elsewhere are confounded. Only synthesis across candidates (Fusion) consistently improves over single-sample baselines, yet still recovers only ~40% of available quality. The candidate pool is not the bottleneck - choosing from it is.
