Search papers, labs, and topics across Lattice.
This paper introduces Counter-Evidence Verification (CoEV), a novel framework for detecting and correcting hallucinations in medical vision-language models (VLMs) by performing bidirectional verification between textual assertions and visual evidence. CoEV operates without the need for retraining, utilizing a four-quadrant diagnostic map to assess the factuality of text in relation to its corresponding visual evidence. Experimental results demonstrate that CoEV significantly outperforms existing hallucination detection methods, improving PR-AUC and ROC-AUC by 3.0% and 3.9% respectively, while also enhancing Micro-F1 scores and reducing hallucination rates in medical report generation.
CoEV not only detects hallucinations in medical VLMs but also corrects them in real-time, enhancing diagnostic reliability without retraining.
Vision-Language models (VLMs) reliability in medical diagnosis is challenged by trust-undermining hallucinations. Existing hallucination detection approaches mainly focus on identifying factual inconsistencies between generated text and reference data. While some studies analyze where models attend in images, they seldom verify whether such attention truly reflects the visual evidence supporting the generated text. To address this gap, we propose Co}unter-Evidence Verification (CoEV), a training-free plug-and-play framework that detects and corrects hallucinations through evidence-based factual consistency verification. CoEV performs bidirectional verification between textual assertions and visual evidence, testing whether each statement is supported by its corresponding evidence region, and assigns each statement into a four-quadrant diagnostic map capturing combinations of text factuality and visual grounding. CoEV detects hallucinated content and serves as a post hoc refinement tool, correcting hallucinations without retraining. Extensive experiments on four medical datasets show that CoEV combats hallucinations in VLMs.For hallucination detection, CoEV consistently outperforms existing methods, improving average PR-AUC and ROC-AUC by 3.0% and 3.9% absolute points respectively, with notable gains of up to 18.5% in specific VQA scenarios. For hallucination correction, it improves Micro-F1 by up to 12.5%, reduces hallucination rates by over 11.9% on medical report generation, and also boosts medical VQA accuracy. These results show that CoEV enables reliable detection and correction of hallucinations, providing clinicians with dependable, evidence-based cues for diagnosis. Code will be released upon acceptance.
NUS