Search papers, labs, and topics across Lattice.
The paper introduces the Multi-Sequence Verifier (MSV), a novel approach to parallel test-time scaling that jointly processes multiple candidate solutions generated by a large language model to improve answer selection and reduce inference latency. MSV enhances verifier calibration by modeling interactions between candidate solutions, leading to improved best-of-N selection performance. A streaming MSV variant enables a new early-stopping framework that fully leverages parallel decoding, achieving the same accuracy with significantly reduced latency compared to isolated solution scoring.
By jointly processing candidate solutions, the Multi-Sequence Verifier slashes LLM latency in parallel test-time scaling by 50% while maintaining accuracy.
Parallel test-time scaling, which generates multiple candidate solutions for a single problem, is a powerful technique for improving large language model performance. However, it is hindered by two key bottlenecks: accurately selecting the correct solution from the candidate pool, and the high inference latency from generating many full solutions. We argue that both challenges are fundamentally linked to verifier calibration. A well-calibrated verifier not only improves answer selection, but also enables early-stopping strategies to reduce latency. However, existing verifiers are limited as they score each candidate in isolation, overlooking rich contextual information across the set of candidates. To address this, we introduce the Multi-Sequence Verifier (MSV), the first verifier designed to jointly process all candidate solutions and model their interactions. MSV achieves improved calibration, which directly enhances best-of-N selection performance. We further introduce a streaming MSV variant that empowers a novel early-stopping framework. Our novel framework fully leverages parallel decoding, which contrasts with the existing multi-sequence early exit works that decode sequences one by one and thus incur significant latency. In this novel setting, MSV can achieve the same target accuracy with around half the latency that would be required with its counterpart that scores each solution in isolation.
