Search papers, labs, and topics across Lattice.
The paper addresses reward sparsity in reinforcement learning (RL) for medical visual grounding, where localizing small or ambiguous regions of interest is challenging. They introduce MedLoc-R1, a performance-aware reward scheduling framework that dynamically adjusts the reward criterion based on model performance during training. By progressively tightening the reward schedule from dense to stricter signals, MedLoc-R1 improves localization accuracy and training stability compared to GRPO-based baselines across three medical visual grounding benchmarks.
Overcome reward sparsity in medical visual grounding by dynamically tightening reward criteria based on model performance, leading to improved localization accuracy and training stability.
Medical visual grounding serves as a crucial foundation for fine-grained multimodal reasoning and interpretable clinical decision support. Despite recent advances in reinforcement learning (RL) for grounding tasks, existing approaches such as Group Relative Policy Optimization~(GRPO) suffer from severe reward sparsity when directly applied to medical images, primarily due to the inherent difficulty of localizing small or ambiguous regions of interest, which is further exacerbated by the rigid and suboptimal nature of fixed IoU-based reward schemes in RL. This leads to vanishing policy gradients and stagnated optimization, particularly during early training. To address this challenge, we propose MedLoc-R1, a performance-aware reward scheduling framework that progressively tightens the reward criterion in accordance with model readiness. MedLoc-R1 introduces a sliding-window performance tracker and a multi-condition update rule that automatically adjust the reward schedule from dense, easily obtainable signals to stricter, fine-grained localization requirements, while preserving the favorable properties of GRPO without introducing auxiliary networks or additional gradient paths. Experiments on three medical visual grounding benchmarks demonstrate that MedLoc-R1 consistently improves both localization accuracy and training stability over GRPO-based baselines. Our framework offers a general, lightweight, and effective solution for RL-based grounding in high-stakes medical applications. Code \& checkpoints are available at \hyperlink{}{https://github.com/MembrAI/MedLoc-R1}.
