Search papers, labs, and topics across Lattice.
This paper investigates the phenomenon of over-sampling in language models during test-time scaling, revealing that excessive sampling can lead to worse answers due to selection errors. It introduces the concepts of the modal ceiling and correlation ceiling, which describe the limits of effective sampling in producing reliable outputs. The key finding is that after a certain threshold, additional samples do not improve answer selection and may even exacerbate mistakes, emphasizing the importance of determining an optimal sampling limit.
Over-sampling can lead language models to confidently select incorrect answers, revealing a critical limit to test-time scaling that researchers must heed.
People overthink; language models over-sample, and the extra effort can talk both into a worse answer. Reasoning systems answer a hard question by sampling it many times (test-time scaling), and the more they draw, the more often a correct answer turns up somewhere, so coverage, the fraction of problems with at least one correct try, climbs and appears to be progress. But a deployed system must return one answer, and choosing it, not knowing which try is right, is selection; selection is capped, and past a point extra samples only make the model surer of a confident mistake, even as every draw adds cost. The gap between climbing coverage and stalled selection, the identifiability gap, is the answer a model can produce but not pick. So the real question is not whether to sample but how far, and the answer is: not far. For picking an answer, the vote has already settled within a few dozen draws, the modal ceiling; for scoring a benchmark, sooner still, the correlation ceiling. Beyond that, extra draws cost compute and add nothing, and can even make the answer worse. This paper turns the cutoff into a single number, the effective number of samples, that any sampling run already reveals. The bottleneck is recognizing a right answer, not generating one.
