Search papers, labs, and topics across Lattice.
This paper introduces Disco-LoRA, a novel framework for multi-concept video customization that disentangles and recombines content, style, and motion in a systematic manner. By employing an Iterative Dual-LoRA Disentanglement Framework, the authors address the challenges of controlling these concepts simultaneously and propose a Z-score-based statistical regularization to harmonize weight distributions across layers. The extensive experiments demonstrate that Disco-LoRA significantly enhances the quality of controllable text-to-video generation, outperforming existing methods in preserving the desired attributes of generated videos.
Disco-LoRA achieves unprecedented control over content, style, and motion in video generation, setting a new benchmark for multi-concept customization.
Video customization based on Text-to-Video (T2V) models aims to learn specific features from reference data to generate controllable videos. While significant strides have been made in image stylization and video motion customization, simultaneously controlling multiple concepts, such as content, style, and motion, remains a major challenge. In this work, we systematically define the task of multi-concept video customization, which requires the joint control of content, style, and motion. To facilitate research in this area, we construct a comprehensive benchmark and propose Disco-LoRA, a unified framework designed to tackle this problem by disentangling and flexibly recombining different concepts in two stages: (1) We decompose the objective into two sub-tasks: Content-Style and Content-Motion. Each sub-task is addressed using our Iterative Dual-LoRA Disentanglement Framework, which effectively disentangles distinct concepts within the data. (2) We identify layer-wise weight trends as crucial for LoRA identity, while weight magnitudes dictate composability. To harmonize these scales, we propose a Z-score-based statistical regularization that aligns weight distributions, preserving layer-wise trends while minimizing interference between different LoRAs. Extensive experiments show that Disco-LoRA excels in multi-concept video customization, effectively preserving appearance, style, and motion for controllable text-to-video generation.
