Search papers, labs, and topics across Lattice.
This paper introduces Best-of-Evidence (BoE), an innovative framework for improving model outputs in vision-language tasks by selecting candidates based on partial verification rather than relying solely on complete evaluations. By utilizing a signed candidate-factor graph and allocating a limited budget for evidence actions, BoE enhances the selection process while maintaining a fixed pool of candidates. Experimental results across four medical VQA settings demonstrate that BoE can effectively improve selection outcomes and address failures in the Best-of-N (BoN) approach when evidence is both reliable and relevant, revealing critical limits in candidate generation and channel quality.
BoE transforms candidate selection by leveraging partial verification, significantly enhancing outcomes in vision-language tasks where complete evaluations are unattainable.
BoN improves model outputs by sampling several candidates and selecting one with a proxy score, but it assumes that complete candidates can be evaluated reliably. Many vision-language tasks instead provide only partial verification: a finding, span, value, region, or relation may be checkable even when no dependable whole-response verifier exists. Moreover, the same claim may recur across candidates with opposing stances, allowing one observation to support part of the pool and contradict another. We introduce Best-of-Evidence (BoE), an inference-time selection framework that keeps the BoN candidate pool fixed, represents reusable claims with a signed candidate--factor graph, and allocates a limited budget to evidence actions that can change the final choice. BoE formalizes selection under partial verification and provides a practical score-based controller, with the zero-budget case recovering the underlying BoN decision. Theoretically, we show that residual evidence capacity limits any evidence-driven improvement and that shared factor queries can achieve an O(log K) versus 螛(K) query separation in a factor-code model. Common-ledger experiments on four medical VQA settings show that BoE can improve fixed-pool selection and rescue some BoN failures when evidence is reliable, contrastive, and decision-relevant, while also revealing the channel-quality and candidate-generation limits that prevent universal gains.
