Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces POIROT, a novel protocol that utilizes the inherent epistemic diversity of agents within a multi-agent system (LLM-MAS) to detect failures and hallucinations, addressing a critical gap in safety for AI deployment. By repurposing the system's own agents as a diagnostic layer, POIROT significantly outperforms traditional single-LLM evaluator baselines, achieving a notable improvement in fault detection that scales with problem complexity and agent count. The findings suggest that internal oversight by agents can effectively enhance safety without relying on external evaluators, thereby aligning with emerging regulatory demands.
POIROT reveals that multi-agent systems can self-audit for failures, outperforming traditional evaluation methods and paving the way for safer AI deployments.
Orchestrating Large Language Models into Multi-Agent Systems (LLM-MAS) has unlocked remarkable reasoning capabilities, yet emergent failures and hallucinations that resist characterisation block their deployment in safety-critical domains -- a gap made legally untenable by emerging AI regulation. Existing evaluation paradigms share a common flaw: centralised judgment creates single points of failure and demands domain-specific expertise. Here we present POIROT, a protocol that repurposes a system's own agents as its diagnostic layer, leveraging the epistemic diversity already present in the architecture. Across evaluated settings, POIROT outperforms single-LLM evaluator baselines, with gains that scale with problem complexity (OR = 1.60, $p = 0.008$), agent count, and fault dimensionality, persisting under compound fault conditions. These results demonstrate that safety oversight need not be externalised: the agents executing a role carry sufficient collective intelligence to audit it. We release POIROT as an open-source library alongside BLAME, a benchmark for fault attribution in safety-critical multi-agent systems.