Search papers, labs, and topics across Lattice.
This study investigates the dynamics of answer selection in multi-agent systems (MAS) by analyzing how the presence and evaluation of candidates influence the final reported answers. Through a comprehensive analysis of 15,336 questions and the replay of 81,390 fixed candidate pools, the authors reveal that even when correct answers are available, systems can still converge on incorrect responses due to the interplay of candidate generation and selection pressure from an LLM judge. The findings indicate that enhancing judge reliability and adjusting answer selection rules can significantly improve accuracy, demonstrating that the effectiveness of candidate generation is contingent on the recognition and frequency of correct answers.
Even when correct answers exist, multi-agent systems often report wrong answers due to the dynamics of candidate generation and selection pressure from LLM judges.
Multi-agent systems (MAS) sometimes already have the potential to answer correctly, but still report a wrong answer. Explaining this outcome is difficult because generation, communication and final answer-selection rules usually change simultaneously. We conceptualize multi-agent reasoning as an evolutionary pipeline of candidate generation, peer communication and terminal selection, wherein consensus without quality control can exhibit patterns of memetic drift. We study two questions: (1) when an LLM judge provides effective selection pressure by supplying a signal of answer correctness for candidates generated in a multi-agent system, and (2) when using that signal improves the reported answer. To map judge reliability, we analysed 15,336 questions from MMLU-Pro, GPQA, MedXpertQA and MuSR, with Humanity's Last Exam analysed separately. To test these rules, we replayed 81,390 fixed candidate pools drawn from 16,278 questions across five benchmarks. We report three findings. (1) A correct answer is often already present among the generated candidates, but the system can still converge on and report a wrong answer. (2) Judge reliability is not a fixed trait of the model, but varies with the task, the generator and how rare the correct answer is. (3) Combining answer frequency with the judge's evaluation changed only the final answer-selection rule and raised accuracy from 63.82% to 70.82-70.95%, primarily by rescuing correct answers that were outnumbered by popular errors. In the systems studied here, the value of generating more candidates depends on whether those extra samples make correct answers present, frequent or recognisable. By isolating generation, recognition and selection, these findings establish a diagnostic basis for designing multi-agent architectures that protect generated correct answers from being lost.
