Track the frontier, from condensed matter to cosmology.
Search papers, labs, and topics across Lattice.
To resolve semantic thrashing—where append-only working memory causes attention collapse during iterative video reasoning—the authors develop VideoLoop, a multimodal agent architecture that decouples external storage from active working context. By using an inner loop to retrieve from an unbounded filesystem and rewrite a bounded working memory after each outer-loop reasoning step, the agent continuously strips accumulated noise while retaining salient evidence. This dual-loop mechanism improves long-form video understanding across four LVLM backbones by an average of 4.2% on VideoMME (long) and powers Gemini 3.1 Pro to top-tier scores of 88.3% on VideoMME and 80.9% on LongVideoBench.
Append-only memory guarantees attention collapse in long-horizon video reasoning, but actively rewriting a bounded working memory from an external filesystem boosts hard-subset task accuracy by over 20%.
Long-form video understanding requires multimodal agents to iteratively gather evidence over many reasoning steps. However, most existing agentic methods suffer from semantic thrashing: as append-only working memory grows, attention to key evidence collapses, and the agent loses access to what it has already found. First, we provide a structural argument showing that append-only memory can incorporate newly observed target evidence, but cannot remove accumulated noise or prevent ordered context growth without a rewrite operator. Second, motivated by this analysis, we propose VideoLoop, a multimodal agent with two coupled loops. The outer loop reasons over the video and the inner loop, after each step, retrieves artifacts from an unbounded filesystem of past observations and intermediate analysis, and rewrites a bounded working memory. Extensive experiments demonstrate the effectiveness of VideoLoop, which improves four popular LVLM backbones in a plug-and-play manner, with an average gain of 4.2% points over baseline on VideoMME (long). Further analysis of working memory suggests that VideoLoop mitigates semantic thrashing: on the hardest quarter of VideoMME (long) questions, a blind judge that reads only the agent's context answers 81.1% correctly, versus 60.9% for the append-only agent. With Gemini 3.1 Pro, VideoLoop reaches 88.3% on VideoMME (long), 88.8% on VideoMMMU, and 80.9% on LongVideoBench (long).
Microsoft Research