Search papers, labs, and topics across Lattice.
This paper introduces EVADE, an innovative method designed to enhance the reliability of medical vision-language models (VLMs) by improving their calibration and reducing overconfidence without retraining. EVADE operates by localizing diagnostically relevant regions in images and re-evaluating responses based on both the full image and zoomed views, only committing to answers when both agree. Experimental results demonstrate that EVADE achieves a significant reduction in expected calibration error by up to 45% while maintaining accuracy, outperforming existing methods like chain-of-thought and self-verification in key metrics.
EVADE uniquely enhances the reliability of medical VLMs by verifying diagnostic consistency across different image views, achieving up to 45% better calibration without retraining.
Medical vision-language models (VLMs) can achieve high accuracy but remain unreliable: they are systematically overconfident, benefit little from test-time reasoning, and lack the ability to reliably calibrate trust in their own responses. We introduce EVADE (Evidence-Verified Agentic Diagnosis with Escape), an inferential, non-training method that enhances the safety of deploying a single frozen VLM. EVADE responds and, when uncertain, localises the region most diagnostically relevant, re-answers on a zoomed view, and commits only when both the entire image and the zoomed view responses agree; otherwise, it abstains. To directly address verification hallucination in single-model self-checking, our main idea is to verify gate consistency across different image views rather than re-reading the model's own text. Experimental evaluation on VQA-RAD, SLAKE, and PathVQA using Qwen2.5-VL-7B reports that EVADE is the only method that simultaneously improves both calibration and selective risk while maintaining accuracy, reducing expected calibration error (ECE) by up to 45% compared to zero-shot. Chain-of-thought, self-consistency, and self-verification all fail at least one axis. A grounding analysis reports that self-proposed regions perform better at diagnostic structure localisation than centres or random crops. However, a 7B VLM cannot use this localisation to revise answers. Therefore, reliability gains come from the consistency gate and calibrated abstention.
