Search papers, labs, and topics across Lattice.
This study investigates how large language models (LLMs) revise their answers in multi-agent systems, focusing on the influence of peer responses and authority cues. The researchers found that LLMs are more susceptible to being misled by consensus than they are to correcting their mistakes, with authority labels exacerbating this issue. Notably, common reasoning interventions like chain-of-thought and reflection do not effectively mitigate harmful revisions while maintaining beneficial ones, highlighting the need for verification mechanisms in multi-agent LLM interactions.
Peer agreement can mislead LLMs more easily than it can correct them, raising serious concerns for multi-agent systems.
Large language models are increasingly used in multi-agent systems, where they see and respond to other agents' answers. A key risk is conformity: a model may abandon its own answer simply because others agree on a different one. Prior studies show that LLMs often revise toward a majority answer, but it remains unclear whether these revisions help correct mistakes as often as they introduce new errors. In this paper, we conduct a controlled study in which an LLM first answers a question, then sees simulated peer responses before making a final decision. We manipulate two social cues: consensus structure and authority labels assigned to peers, and measure how they influence beneficial and harmful revisions. Across four open-weight LLMs and seven QA datasets, we find that peer agreement makes it much easier to mislead initially correct models than to correct initially wrong ones. Authority labels make models more likely to choose the endorsed answer, regardless of whether it is correct. More concerningly, generic reasoning interventions such as chain-of-thought and reflection do not reliably reduce harmful revision while preserving beneficial revision. These findings suggest that multi-agent LLM systems should verify peer answers rather than simply aggregate them.
