Search papers, labs, and topics across Lattice.
This paper introduces VicEdit, a novel framework for video editing that utilizes Visual In-context Editing, allowing for the integration of visual guidance alongside textual instructions. By curating the VicEdit-400K dataset and employing Modality-Adaptive Semantic Distillation, the authors enhance the editing process with high-quality visual references, resulting in improved fidelity and semantic consistency. Extensive evaluations reveal that VicEdit outperforms existing methods in both basic instruction editing and complex visual tasks, marking a significant advancement in video editing capabilities.
Visual in-context learning transforms video editing by seamlessly integrating visual cues with textual instructions, achieving state-of-the-art results.
Despite progress in instruction-based video editing, unimodal textual instructions inherently struggle to convey fine-grained textures and complex dynamics. To bridge this perceptual gap, we propose Visual In-context Editing, a new paradigm elevating video editing from textual instructions to multi-modal visual guidance encompassing single image, image pair, and video pair. To facilitate this paradigm, we curate VicEdit-400K, the first large-scale dataset for visual in-context video editing. We develop an automated pipeline to generate 400K high-quality samples across ten task types, ensuring superior visual fidelity and semantic consistency through multi-dimensional filtering. Leveraging this foundation, we introduce VicEdit, a unified framework to bridge visual and textual contexts. To adaptively extract editing semantics from heterogeneous references, we design Modality-Adaptive Semantic Distillation, which produces modality-specific semantic tokens from visual references. These tokens are then synergistically integrated with textual instructions through Dual-Context Injection, enabling the generation process to benefit from both visual and textual signals. Extensive evaluations on VicEditBench demonstrate that VicEdit achieves state-of-the-art performance across both basic instruction editing and visual in-context editing tasks, establishing visual in-context learning as a powerful and controllable paradigm for video editing.
