Search papers, labs, and topics across Lattice.
This paper investigates the performance difference between parallel and sequential sampling strategies in Large Reasoning Models (LRMs) on math and coding tasks. Through experiments with Qwen3, DeepSeek-R1 distilled models, and Gemini 2.5, the authors find that parallel sampling outperforms sequential sampling, even though sequential sampling should have more representation power due to its iterative refinement process. The study concludes that the primary reason for this gap is the reduced exploration in sequential sampling, rather than the aggregator operator or the increased context length.
Seemingly counterintuitively, iterative refinement via sequential sampling hurts the performance of large reasoning models, primarily due to a lack of exploration compared to parallel sampling.
Large Reasoning Models (LRMs) have shown remarkable performance on challenging questions, such as math and coding. However, to obtain a high quality solution, one may need to sample more than once. In principal, there are two sampling strategies that can be composed to form more complex processes: sequential sampling and parallel sampling. In this paper, we first compare these two approaches with rigor, and observe, aligned with previous works, that parallel sampling seems to outperform sequential sampling even though the latter should have more representation power. To understand the underline reasons, we make three hypothesis on the reason behind this behavior: (i) parallel sampling outperforms due to the aggregator operator; (ii) sequential sampling is harmed by needing to use longer contexts; (iii) sequential sampling leads to less exploration due to conditioning on previous answers. The empirical evidence on various model families and sizes (Qwen3, DeepSeek-R1 distilled models, Gemini 2.5) and question domains (math and coding) suggests that the aggregation and context length do not seem to be the main culprit behind the performance gap. In contrast, the lack of exploration seems to play a considerably larger role, and we argue that this is one main cause for the performance gap.
