Search papers, labs, and topics across Lattice.
The paper introduces ArgEval, a framework that uses argumentation frameworks (AFs) to evaluate general decision options for LLMs, moving beyond instance-specific reasoning. ArgEval systematically maps decision spaces, builds option ontologies, and constructs AFs for each option, enabling explainable recommendations and global contestability. The framework's effectiveness is demonstrated in the context of treatment recommendation for glioblastoma, showing alignment with clinical practice.
LLMs can now offer globally contestable decision support by systematically mapping decision spaces into argumentation frameworks, allowing users to challenge the underlying logic, not just individual outputs.
Large language models (LLMs) exhibit strong general capabilities, but their deployment in high-stakes domains is hindered by their opacity and unpredictability. Recent work has taken meaningful steps towards addressing these issues by augmenting LLMs with post-hoc reasoning based on computational argumentation, providing faithful explanations and enabling users to contest incorrect decisions. However, this paradigm is limited to pre-defined binary choices and only supports local contestation for specific instances, leaving the underlying decision logic unchanged and prone to repeated mistakes. In this paper, we introduce ArgEval, a framework that shifts from instance-specific reasoning to structured evaluation of general decision options. Rather than mining arguments solely for individual cases, ArgEval systematically maps task-specific decision spaces, builds corresponding option ontologies, and constructs general argumentation frameworks (AFs) for each option. These frameworks can then be instantiated to provide explainable recommendations for specific cases while still supporting global contestability through modification of the shared AFs. We investigate the effectiveness of ArgEval on treatment recommendation for glioblastoma, an aggressive brain tumour, and show that it can produce explainable guidance aligned with clinical practice.
