Search papers, labs, and topics across Lattice.
The paper introduces GLEAN, a framework for verifying LLM-powered agents in high-stakes decision-making scenarios by grounding evidence accumulation in expert-curated guidelines. GLEAN evaluates step-wise alignment with domain guidelines, aggregates these ratings into surrogate features, and calibrates correctness probabilities using Bayesian logistic regression. Experiments on clinical diagnosis using the MIMIC-IV dataset demonstrate that GLEAN outperforms existing verifiers by 12% in AUROC and 50% in Brier score reduction, while also being recognized as useful by clinicians.
LLM agents in high-stakes domains can be verified more reliably by accumulating evidence grounded in expert guidelines, achieving a 12% AUROC improvement and 50% Brier score reduction over existing methods.
As LLM-powered agents have been used for high-stakes decision-making, such as clinical diagnosis, it becomes critical to develop reliable verification of their decisions to facilitate trustworthy deployment. Yet, existing verifiers usually underperform owing to a lack of domain knowledge and limited calibration. To address this, we establish GLEAN, an agent verification framework with Guideline-grounded Evidence Accumulation that compiles expert-curated protocols into trajectory-informed, well-calibrated correctness signals. GLEAN evaluates the step-wise alignment with domain guidelines and aggregates multi-guideline ratings into surrogate features, which are accumulated along the trajectory and calibrated into correctness probabilities using Bayesian logistic regression. Moreover, the estimated uncertainty triggers active verification, which selectively collects additional evidence for uncertain cases via expanding guideline coverage and performing differential checks. We empirically validate GLEAN with agentic clinical diagnosis across three diseases from the MIMIC-IV dataset, surpassing the best baseline by 12% in AUROC and 50% in Brier score reduction, which confirms the effectiveness in both discrimination and calibration. In addition, the expert study with clinicians recognizes GLEAN's utility in practice.
