Search papers, labs, and topics across Lattice.
The paper introduces MedVR, a reinforcement learning framework that enables annotation-free visual reasoning for medical Vision-Language Models (VLMs). MedVR uses Entropy-guided Visual Regrounding (EVR) to direct exploration based on model uncertainty and Consensus-based Credit Assignment (CCA) to distill pseudo-supervision from rollout agreement. Experiments on medical VQA benchmarks demonstrate that MedVR significantly outperforms existing models by learning to reason directly with visual evidence.
Medical VLMs can achieve state-of-the-art visual reasoning without any human annotations, thanks to a novel reinforcement learning framework that leverages model uncertainty and rollout agreement.
Medical Vision-Language Models (VLMs) hold immense promise for complex clinical tasks, but their reasoning capabilities are often constrained by text-only paradigms that fail to ground inferences in visual evidence. This limitation not only curtails performance on tasks requiring fine-grained visual analysis but also introduces risks of visual hallucination in safety-critical applications. Thus, we introduce MedVR, a novel reinforcement learning framework that enables annotation-free visual reasoning for medical VLMs. Its core innovation lies in two synergistic mechanisms: Entropy-guided Visual Regrounding (EVR) uses model uncertainty to direct exploration, while Consensus-based Credit Assignment (CCA) distills pseudo-supervision from rollout agreement. Without any human annotations for intermediate steps, MedVR achieves state-of-the-art performance on diverse public medical VQA benchmarks, significantly outperforming existing models. By learning to reason directly with visual evidence, MedVR promotes the robustness and transparency essential for accelerating the clinical deployment of medical AI.
Tsinghua AI