Search papers, labs, and topics across Lattice.
This paper addresses the limitations of existing post-training alignment methods for Large Vision-Language Models (LVLMs) in the medical domain, specifically targeting issues like factual inconsistencies and poor visual grounding. The authors introduce a novel fine-grained preference optimization framework that employs a bidirectional token-wise KL regularizer and a visual-contrastive grounding objective to enhance clinical correctness while maintaining linguistic style. Experimental results demonstrate significant improvements in model performance across various medical imaging tasks and clinical text generation benchmarks, highlighting the importance of precise alignment in medical applications.
Fine-grained preference optimization can dramatically enhance the clinical accuracy of LVLMs by correcting only the clinically erroneous spans while preserving linguistic style.
Large Vision-Language Models (LVLMs) have achieved strong performance across medical imaging tasks, yet they remain prone to factual inconsistencies, poor visual grounding, and misalignment with clinically meaningful feedback. Existing post-training alignment approaches, including Direct Preference Optimization (DPO) and its variants, face three critical limitations in the medical domain: (1) sequence-level reward signals treat clinically critical tokens identically to generic filler text; (2) reliance on static supervised fine-tuning references as preferred responses introduces an off-policy distribution shift, steering optimization toward stylistic artifacts over clinical correctness; and (3) alignment objectives lack explicit visual grounding constraints, leaving models insensitive to subtle yet diagnostically decisive pathological features. Our method leverages a bidirectional token-wise KL regularizer alongside a visual-contrastive grounding objective that pairs clean and lesion-corrupted images to penalize responses generated without adequate visual evidence. Together, these components form a fine-grained, on-policy alignment framework that constructs preference pairs by minimally editing model-generated outputs, correcting only clinically erroneous spans while preserving the original linguistic style. Extensive experiments across medical imaging tasks and clinical text generation benchmarks validate the effectiveness of our approach.
