Search papers, labs, and topics across Lattice.
AgentCVR is introduced as a multi-agent framework for Cross-Video Reasoning (CVR), addressing the limitations of single-pass MLLMs in handling distributed evidence across multiple videos. The framework uses a Master Agent to coordinate Visual and Audio Agents for targeted evidence extraction. To improve training efficiency, the authors propose Script-Simulated RL, which uses LLM-generated semantic scripts and a text-based simulator to optimize agent policies, achieving state-of-the-art performance on a CVR benchmark.
LLMs can actively reason across multiple videos at near state-of-the-art performance by iteratively dispatching specialized visual and audio agents, even when trained with a lightweight text-based simulator.
Cross-Video Reasoning (CVR) has emerged as a critical frontier in multimodal intelligence, requiring models to retrieve, align, and aggregate evidence distributed across multiple videos. Current Multimodal Large Language Models (MLLMs) often struggle with CVR, as simple single-pass strategies encode multiple videos into a shared compressed context, potentially obscuring rare but critical evidence. In this paper, we propose AgentCVR, a multi-agent framework that treats CVR as an active evidence-acquisition task. AgentCVR employs a Master Agent to iteratively coordinate specialized Visual and Audio Agents for targeted evidence extraction. To ensure efficient training, we introduce Script-Simulated RL, which optimizes the agent's policy with LLM-generated semantic scripts and a lightweight text-based simulator, bypassing costly multimodal inference during online exploration. Experimental results on a comprehensive CVR benchmark show that AgentCVR outperforms single-pass baselines and achieves comparable performance to state-of-the-art closed-source systems, particularly in complex cross-video alignment and localization. To ensure reproducibility, our code is available at https://github.com/wang-jh24/AgentCVR.
