Search papers, labs, and topics across Lattice.
This paper introduces the Mixture of Debaters (MoD), a novel framework that enables dynamic self-debate within a single model, overcoming the limitations of static architectures and high computational costs in existing multi-agent debate systems. By employing the Mixture-of-Experts paradigm, MoD implements dual-routing for flexible role allocation, momentum switching to enhance token-level routing, and unified self-debate to maintain diverse debating personas without inter-agent communication. Experimental results show that MoD significantly outperforms traditional multi-agent systems and single-model baselines, achieving 3.7 times lower latency and an 87% reduction in token consumption while enhancing accuracy on multimodal benchmarks.
MoD achieves 3.7x lower latency and 87% less token consumption while outperforming traditional multi-agent debate systems in accuracy.
Existing multi-agent debate frameworks suffer from two critical limitations: they rely on static architectures where agent roles and coordination patterns are fixed at design time, and they require instantiating multiple model copies, incurring substantial computational overhead. We propose Mixture of Debaters (MoD), a unified framework that enables dynamic self-debate within a single model by leveraging the Mixture-of-Experts paradigm. We address three key challenges in adapting MoE for dialectical reasoning: (1) dual-routing that decouples role allocation from process flow, dynamically determining when to debate versus when to synthesize; (2) momentum switching that smooths token-level routing with local context, reducing expert-switch jitter; and (3) unified self-debate that encapsulates diverse debating personas into lightweight expert modules, eliminating inter-agent communication while preserving behavioral diversity. Extensive experiments on multimodal benchmarks demonstrate that MoD outperforms both single-model baselines and conventional multi-agent systems, achieving superior accuracy with 3.7x lower latency and 87% reduction in token consumption.The source code can be accessed at https://github.com/YongLD/MoD.
