Search papers, labs, and topics across Lattice.
This study investigates the dynamics of heterogeneous LLM debate by analyzing how diverse peer interactions influence the revision behavior of honest agents in the presence of adversarial peers. Through a series of experiments across four model families and three reasoning benchmarks, the authors find that the inclusion of an honest heterogeneous peer significantly reduces harmful revisions, while the presence of an adversarial peer reverses these gains. Specifically, in the Llama-3.1-70B model on MATH-hard tasks, harmful revision rates drop from 89% to 35% with an honest peer, but return to 90% when an adversarial peer is involved, highlighting the dual role of heterogeneity as both a vulnerability and a potential defense mechanism.
Honest heterogeneous peers can drastically reduce harmful revisions in LLM debates, but adversarial peers can completely undermine these benefits.
Heterogeneous LLM debate is motivated by the promise that diverse peers correct one another, but the same exchange that carries correction also carries adversarial influence. We measure which dominates by tracking how a heterogeneous peer changes the honest agents' revision behavior: how often they change their answer, and whether the change is corrective or harmful. We compare matched panels (homogeneous baseline, honest-mixed, and adversarial-mixed) and contaminated panels in which a malicious same-family peer is already present, spanning four model families and three reasoning benchmarks. An honest heterogeneous peer sharply lowers harmful revision, and an adversarial one reverses it. For Llama-3.1-70B defenders on MATH-hard, the honest-slot harmful-revision rate falls from 89% in the homogeneous panel to 35% with an honest peer, and an adversarial peer returns it to 90%. The conditional rate hides this damage on weak defenders, but the end-of-debate flip rate exposes it. The pattern keeps its sign across families and benchmarks while its magnitude varies with the defender-benchmark regime. We also measure the effects when an adversarial same-family peer is already present: an honest heterogeneous peer lowers both harmful revision and the rate at which initially-correct answers are lost. On the same Llama-3.1-70B setting, the added honest peer cuts the flip rate on initially-correct items from 31% under a same-family adversary to 6%. Heterogeneity is therefore not only an attack surface but, when an adversary is already present, also a defense.
