Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces ROVER, a lightweight plugin for MLLMs that improves grounded multi-image reasoning by routing object-centric visual evidence. ROVER uses a step-specific token triplet to aggregate reasoning context, distill intra-image cues via object-centric differential attention, and route history-aware evidence across objects and images. Integrating ROVER into Qwen2.5-VL-7B and using an interleaved SFT-to-GRPO training pipeline achieves state-of-the-art results on MM-GCoT and VideoEspresso, demonstrating strong transferability.
Achieve state-of-the-art grounded multi-image reasoning by routing visual evidence between objects, without sacrificing holistic scene understanding.
Multimodal Large Language Models (MLLMs) have increasingly localized and interleaved visual evidence for deliberative reasoning. Grounding-based approaches typically focus on regions of interest (RoIs) by injecting cropped image patches or RoI-specific features into the reasoning context. However, such designs can weaken holistic scene understanding and inter-object relations, while incurring decoding costs that scale with the number and size of RoIs. Alternatively, adaptive visual feature selection often requires fine-grained supervision or complex heuristics. To address these limitations, we propose ROVER (Routing Object-centric Visual Evidence for grounded multi-image Reasoning), a lightweight, learnable plugin for efficient global visual evidence routing. Upon each object grounding prediction, ROVER injects a step-specific token triplet to synergistically: (i) aggregate the ongoing reasoning context, (ii) distill intra-image cues into a visual working space via object-centric differential attention, and (iii) route and integrate history-aware evidence across objects and images within this space for subsequent reasoning. We integrate ROVER into Qwen2.5-VL-7B and develop an interleaved SFT-to-GRPO training pipeline. Strictly adhering to the original datasets and evaluation protocols, our method achieves the best performance on MM-GCoT (+4.8% answer accuracy, +14.6% grounding accuracy) and VideoEspresso (+8.6% answer accuracy). The VideoEspresso-trained model demonstrates strong transferability, outperforming the base model by +4.7% on average across diverse benchmarks.