Search papers, labs, and topics across Lattice.
This paper introduces ALIVE, a framework for video editing that enables inserted objects to participate in coherent interactions with existing video content, addressing a key limitation of current methods. By training on a large dataset of curated editing pairs and a vision-language model (VLM) for interaction guidance, ALIVE achieves significant improvements in interaction fidelity and source preservation. The proposed ALIVE-interaction benchmark provides a unified VLM-based evaluation protocol for assessing these crucial aspects of video editing.
Inserted objects can now be made "alive" and interact realistically within videos, solving a long-standing challenge in video editing.
Current video editors can insert objects but often struggle to make them participate in interactions such as being picked up or manipulated. We introduce ALIVE, a framework that makes inserted objects"alive"through coherent interactions with the source video's contents, using an edited first frame and an instruction naming only the added object. We curate 35,800 editing pairs combining 3D-rendered, model-generated, and real-world videos with general editing pairs from ROSE. Each pair differs in the target object's presence while preserving the surrounding action, teaching editors coordinated object behavior and source preservation. We further train a vision-language model (VLM) to predict interaction guidance from the same inputs. We introduce the ALIVE-interaction benchmark to assess interaction fidelity, source preservation, and visual coherence using a unified VLM-based protocol, and evaluate on the general video object insertion benchmark. Without VLM guidance, ALIVE improves Overall over the strongest evaluated baseline by 43.9% and 4.4% on the two benchmarks, respectively. VLM-predicted guidance further improves the ALIVE-interaction score by 0.95 points without additional user inputs.
