Search papers, labs, and topics across Lattice.
This paper challenges the assumption that more temporal observations inherently improve video-language reasoning, arguing instead that reliable reasoning under limited evidence is the core challenge. The authors introduce SAVER, a post-training framework that leverages dense video references to guide sparse-frame inference via reinforcement learning with grounding and reliability-gated rewards. SAVER achieves competitive or superior performance to dense-frame baselines on temporal grounding and video question-answering tasks using significantly fewer frames, demonstrating temporal grounding's efficacy as an evidence-localization proxy.
Sparse video reasoning can outperform dense observation models by learning to extract crucial temporal evidence from limited frames, even without explicit VQA annotations.
Video-language models commonly assume that more temporal observations lead to more reliable reasoning. We question this assumption and argue that the key challenge is not merely processing more video frames efficiently, but learning to reason reliably under limited temporal evidence. We propose SAVER, a dense-to-sparse post-training framework that uses dense video views as training-time references for sparse-frame inference. During reinforcement post-training, paired dense and sparse views are optimized with grounding rewards and a reliability-gated reference reward, encouraging sparse view predictions to preserve task-relevant temporal evidence. Notably, SAVER is trained only on 1,250 randomly sampled temporal grounding examples, without using any video question answering annotations. Across three temporal grounding benchmarks and six video question-answering benchmarks, SAVER consistently improves performance across frame budgets. In particular, SAVER can match or surpass dense-frame Qwen3.5 baselines while using substantially fewer frames. These results show that temporal grounding can serve as an effective evidence-localization proxy for learning sparse video reasoning that transfers to broader video understanding tasks.
