Search papers, labs, and topics across Lattice.
This paper introduces DyCo-RL, a novel approach that enhances Reinforcement Learning with Verifiable Rewards (RLVR) by incorporating dynamic cross-modal coordination to improve visual reasoning in Multimodal Large Language Models (MLLMs). The authors identify a critical breakdown in the ability of MLLMs to alternately extract visual evidence and synthesize textual context during Chain-of-Thought reasoning, which is linked to reasoning failures. By employing the Fisher-Rao geodesic distance to optimize attention shifts and aligning token roles, DyCo-RL significantly boosts the performance of existing RLVR algorithms across multiple benchmarks, demonstrating its effectiveness in visual-centric and mathematical reasoning tasks.
MLLMs often struggle with reasoning due to a failure in dynamic cross-modal coordination, but DyCo-RL fixes this by optimizing attention shifts for better performance.
Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a leading paradigm for enhancing visual reasoning in Multimodal Large Language Models (MLLMs). However, existing RLVR methods optimize primarily for the reasoning outcome, fundamentally overlooking the fine-grained cross-modal coordination required during the generation process. Through token-level analyses and controlled interventions, we reveal that during Chain-of-Thought (CoT) reasoning, MLLMs frequently fail to dynamically alternate between extracting visual evidence and synthesizing textual context-a coordination breakdown that is causally linked to reasoning failures. Motivated by these findings, we propose DyCo-RL, which integrates dynamic cross-modal coordination into RLVR optimization. Specifically, DyCo-RL uses the Fisher-Rao geodesic distance to measure within-modality attention shifts, assigning tokens to either visually-oriented or text-oriented functional roles. It then evaluates the alignment between a token's actual attention allocation and its assigned role, leveraging this score for alignment-guided advantage reweighting during policy optimization. Extensive experiments demonstrate that the algorithm-agnostic DyCo-RL, when applied to Qwen2.5-VL-3B/7B, consistently improves four representative RLVR algorithms across seven benchmarks spanning visual-centric and mathematical reasoning.
