Search papers, labs, and topics across Lattice.
This paper introduces VideoRouter (VR), a novel approach to long-video understanding that organizes video content into a question-agnostic temporal hierarchy, allowing for the coordination of complementary evidence views rather than relying solely on frame selection. By structuring videos into coarse-to-fine segments, VR captures both broad storyline context and fine-grained local details, enabling enhanced reasoning capabilities. Experimental results demonstrate that VR outperforms existing frame selection methods by 2.9 points on the VideoMME benchmark when utilizing the LLaVA-Video-7B backbone, showcasing its effectiveness in integrating global and local reasoning.
Coordinating global and local reasoning in long-video understanding leads to a significant 2.9 point improvement over traditional frame selection methods.
Multimodal Large Language Models (MLLMs) have achieved strong progress in video understanding, yet it remains challenging because the token limitation makes MLLMs difficult to capture temporally sparse evidence. Existing methods typically rely on uniform sampling, or frame selection, but these strategies usually optimize either broad temporal coverage or local relevance, making it difficult to preserve both global storyline context and fine-grained evidence. We propose VideoRouter(VR) that rethinks long-video understanding as coordinating complementary evidence views rather than selecting a single subset of frames. It first organizes each video into a question-agnostic temporal hierarchy, which partition the video into coarse-to-fine temporally coherent segments. In this hierarchy, upper-level nodes capture broad storyline context and event progression, while lower-level nodes preserve fine-grained local details and evidence-bearing moments. This naturally gives rise to two complementary views: a global view for coverage-oriented reasoning and a local view for detail-oriented evidence recovery. We further introduce a verification-guided router to determine which view is better supported by the selected evidence and select the final answer. We validate the effectiveness of the proposed design through extensive experiments, showing that the verification-guided router effectively coordinates global and local reasoning, and that, on VideoMME, our method outperforms state-of-the-art frame selection methods by 2.9 points, respectively, under the LLaVA-Video-7B backbone. We will release the code.
