Search papers, labs, and topics across Lattice.
The paper introduces Symphony, a multi-agent system designed to improve long-form video understanding (LVU) by emulating human cognition patterns through fine-grained subtask decomposition and deep reasoning collaboration enhanced by reflection. Symphony uses a VLM-based grounding approach to analyze LVU tasks and assess the relevance of video segments, improving the ability to locate complex problems. Experiments demonstrate that Symphony achieves state-of-the-art performance on LVBench, LongVideoBench, VideoMME, and MLVU, outperforming previous methods by 5.0% on LVBench.
Symphony's cognitively-inspired multi-agent system significantly boosts long-form video understanding by mimicking human reasoning, achieving state-of-the-art results on multiple benchmarks.
Despite rapid developments and widespread applications of MLLM agents, they still struggle with long-form video understanding (LVU) tasks, which are characterized by high information density and extended temporal spans. Recent research on LVU agents demonstrates that simple task decomposition and collaboration mechanisms are insufficient for long-chain reasoning tasks. Moreover, directly reducing the time context through embedding-based retrieval may lose key information of complex problems. In this paper, we propose Symphony, a multi-agent system, to alleviate these limitations. By emulating human cognition patterns, Symphony decomposes LVU into fine-grained subtasks and incorporates a deep reasoning collaboration mechanism enhanced by reflection, effectively improving the reasoning capability. Additionally, Symphony provides a VLM-based grounding approach to analyze LVU tasks and assess the relevance of video segments, which significantly enhances the ability to locate complex problems with implicit intentions and large temporal spans. Experimental results show that Symphony achieves state-of-the-art performance on LVBench, LongVideoBench, VideoMME, and MLVU, with a 5.0% improvement over the prior state-of-the-art method on LVBench. Code is available at https://github.com/Haiyang0226/Symphony.
