Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces MEDit-Bench, a novel dataset designed to evaluate message-driven narrative video editing by pairing long-form videos with various editing messages and corresponding professional edits. This work highlights the significant impact of narrative intent on editing choices, revealing that different messages lead to substantially different edits even from the same footage. Through experiments with state-of-the-art models, the authors find that while models can achieve reasonable temporal alignment with human edits, they consistently underperform at stricter evaluation criteria, underscoring the challenges in capturing narrative quality.
Narrative intent dramatically alters video edits, yet current models struggle to match human quality, revealing a critical gap in AI video editing capabilities.
Video editing is fundamentally message-driven: even from the same source footage, the selected shots change depending on the narrative the editor wishes to convey. Benchmarks for a closely related task, video summarization, reduce editorial intent to a single, message-agnostic notion of saliency and thus do not account for this diversity. For evaluating message-driven video editing, we present \textbf{MEDit-Bench}, a dataset and benchmark, which pairs long-form videos with multiple editing messages and multiple professionally produced edits per message, demonstrating that different messages yield substantially different edits from the same source. We define an automatic evaluation protocol based on temporal alignment metrics, and find that an LLM-as-a-judge preference, a natural proxy for narrative quality, is unreliable for this task due to severe position bias. We additionally annotate each message with ambiguity and contextfulness scores, and show that both dimensions negatively correlate with model performance, establishing message difficulty as a meaningful stratification factor. Experiments with state-of-the-art MLLMs and reinforcement fine-tuned baselines show that while strong models approach human temporal alignment at lenient thresholds, all models fall behind humans at stricter criteria. A human perceptual study further confirms a large quality gap, with professional human edits remaining consistently preferred over model outputs.