Search papers, labs, and topics across Lattice.
The paper introduces a multi-agent debate framework called the "Social Laboratory" to evaluate emergent social and cognitive dynamics in LLM agents, moving beyond traditional downstream task benchmarks. This framework uses LLM-based agents with distinct personas and incentives debating under the supervision of an LLM moderator. The study reveals a strong tendency for agents to seek consensus, stable psychometric profiles induced by assigned personas, and the significant influence of the moderator's persona on debate outcomes.
LLM agents exhibit a surprising drive for consensus in debates, consistently reaching high semantic agreement even without explicit instructions, revealing an emergent social behavior that demands further investigation.
As Large Language Models (LLMs) transition from static tools to autonomous agents, traditional evaluation benchmarks that measure performance on downstream tasks are becoming insufficient. These methods fail to capture the emergent social and cognitive dynamics that arise when agents communicate, persuade, and collaborate in interactive environments. To address this gap, we introduce a novel evaluation framework that uses multi-agent debate as a controlled"social laboratory"to discover and quantify these behaviors. In our framework, LLM-based agents, instantiated with distinct personas and incentives, deliberate on a wide range of challenging topics under the supervision of an LLM moderator. Our analysis, enabled by a new suite of psychometric and semantic metrics, reveals several key findings. Across hundreds of debates, we uncover a powerful and robust emergent tendency for agents to seek consensus, consistently reaching high semantic agreement ({\mu}>0.88) even without explicit instruction and across sensitive topics. We show that assigned personas induce stable, measurable psychometric profiles, particularly in cognitive effort, and that the moderators persona can significantly alter debate outcomes by structuring the environment, a key finding for external AI alignment. This work provides a blueprint for a new class of dynamic, psychometrically grounded evaluation protocols designed for the agentic setting, offering a crucial methodology for understanding and shaping the social behaviors of the next generation of AI agents. We have released the code and results at https://github.com/znreza/multi-agent-LLM-eval-for-debate.
