Search papers, labs, and topics across Lattice.
This paper introduces VideoAgent, a comprehensive framework designed to enhance video understanding and editing by overcoming limitations in existing automated systems, which typically struggle with long-form content and diverse editing tasks. The framework incorporates automated video shot creation and a multi-agent orchestration system that integrates over thirty specialized editing agents, achieving high orchestration success rates and significant cost reductions. Experimental results indicate that VideoAgent produces professional-quality content that closely matches human performance, with only a 4% difference in quality ratings compared to human-created videos.
VideoAgent not only streamlines video editing with advanced automation but also achieves human-level quality in content creation, setting a new standard in the field.
Video editing has become essential in digital media creation, yet existing automated systems are restricted to short segment processing and domain-specific tasks. They face two critical limitations: i) inability to handle diverse video comprehension and editing operations, and ii) lack of long-video understanding for coherent narrative creation. We propose VideoAgent, an all-in-one agentic framework addressing these challenges through two key innovations. First, we develop automated video shot creation with shot planning agents for coherent narratives and cross-modal retrieval for aligned visual content. Second, we design a multi-agent orchestration framework integrating over thirty specialized editing agents. Intent parsing filters relevant tools while textual-gradient graph optimization assembles complex editing pipelines. Extensive experiments on our newly-proposed VideoEdit benchmark and public datasets demonstrate VideoAgent's superiority over existing multimodal LLMs and agentic systems. VideoAgent achieves 87-95% orchestration success rates while reducing API costs by 60%. Human evaluation across six video categories shows VideoAgent produces professional-quality content approaching human-level performance, with ratings only 4% below human-created videos. We release our code at https://github.com/HKUDS/VideoAgent.
