Search papers, labs, and topics across Lattice.
DynFrame introduces a novel multimodal framework that learns both the temporal window and sampling density for video retrieval within a single autoregressive pass, enabling multi-granularity evidence acquisition. To optimize this process, they propose Segment-Decoupled GRPO (SD-GRPO), which assigns token-level advantages separately to retrieval and answer generation, addressing the issue of uniform credit assignment. Experiments on six benchmarks demonstrate that DynFrame-4B achieves competitive performance with larger baselines, while DynFrame-8B establishes new state-of-the-art results.
Stop wasting compute on redundant retrieval calls: DynFrame learns the optimal frame sampling density *within* each temporal window, slashing context length and boosting performance on complex video understanding tasks.
Recent video multimodal large language models (MLLMs) increasingly couple step-by-step reasoning with on-demand visual evidence retrieval, allowing models to revisit relevant video segments during inference. However, two structural gaps remain in existing thinking-with-video systems. (i) Sampling density is not a learnable decision: existing methods may let the model decide where to look, but the per-window frame rate is largely fixed. As a result, fine-grained evidence is often recovered through repeated retrieval calls, which increases inference context length and training difficulty. (ii) Retrieval and answer generation are usually optimized with a single trajectory-level advantage, so the "where to look" tokens and the "how to answer" tokens receive the same credit even when one is correct and the other is not. To address these gaps, we present DynFrame, a framework that emits the temporal window and the sampling density as native tokens within a single autoregressive pass. This learnable span-density retrieval enables acquiring multi-granularity evidence with a single retrieval step. Based on the above tokenized retrieval interface, we further introduce Segment-Decoupled GRPO (SD-GRPO), which splits each rollout at the retrieval boundary and assigns role-specific token-level advantages, separately crediting the sampling decision and the answer. Trained on the curated DM-CoT-74k and DM-RL-45k, DynFrame-4B is competitive with strong 7B-8B baselines across six benchmarks (NExT-GQA, Charades-STA, ActivityNet-MR, Video-MME, MLVU, LVBench), and DynFrame-8B sets new state-of-the-art on most metrics. Code is available at https://github.com/zhangguanghao523/DynFrame.
DAMO