Search papers, labs, and topics across Lattice.
This paper introduces MAD-ACC, a multi-agent debate framework using a Proponent-Opponent-Judge model to improve argument component classification. By having agents defend conflicting interpretations, the framework exposes logical nuances and reduces classification uncertainty compared to single-agent methods. Experiments on the UKP Student Essays corpus show MAD-ACC achieves a Macro F1 score of 85.7%, outperforming single-agent baselines without domain-specific training, while also providing explainable reasoning through debate transcripts.
Multi-agent debate unlocks significantly better argument classification from LLMs, even without fine-tuning, by surfacing hidden ambiguities that single-agent models miss.
Argument Mining (AM) is a foundational technology for automated writing evaluation, yet traditional supervised approaches rely heavily on expensive, domain-specific fine-tuning. While Large Language Models (LLMs) offer a training-free alternative, they often struggle with structural ambiguity, failing to distinguish between similar components like Claims and Premises. Furthermore, single-agent self-correction mechanisms often suffer from sycophancy, where the model reinforces its own initial errors rather than critically evaluating them. We introduce MAD-ACC (Multi-Agent Debate for Argument Component Classification), a framework that leverages dialectical refinement to resolve classification uncertainty. MAD-ACC utilizes a Proponent-Opponent-Judge model where agents defend conflicting interpretations of ambiguous text, exposing logical nuances that single-agent models miss. Evaluation on the UKP Student Essays corpus demonstrates that MAD-ACC achieves a Macro F1 score of 85.7%, significantly outperforming single-agent reasoning baselines, without requiring domain-specific training. Additionally, unlike "black-box" classifiers, MAD-ACC's dialectical approach offers a transparent and explainable alternative by generating human-readable debate transcripts that explain the reasoning behind decisions.
