Search papers, labs, and topics across Lattice.
The paper introduces Tri-Prompting, a two-stage training framework for video diffusion models that unifies control over scene composition, multi-view consistent subject customization, and motion control. It uses a dual-condition motion module driven by 3D tracking points for background and downsampled RGB cues for foreground, along with a ControlNet scale schedule for balancing controllability and realism. Experiments show Tri-Prompting surpasses specialized baselines in multi-view subject identity, 3D consistency, and motion accuracy, enabling new workflows like 3D-aware subject insertion and manipulation.
Finally, a video diffusion model that lets you precisely control scene, subject, and motion *together*, opening the door to truly customizable AI video creation.
Recent video diffusion models have made remarkable strides in visual quality, yet precise, fine-grained control remains a key bottleneck that limits practical customizability for content creation. For AI video creators, three forms of control are crucial: (i) scene composition, (ii) multi-view consistent subject customization, and (iii) camera-pose or object-motion adjustment. Existing methods typically handle these dimensions in isolation, with limited support for multi-view subject synthesis and identity preservation under arbitrary pose changes. This lack of a unified architecture makes it difficult to support versatile, jointly controllable video. We introduce Tri-Prompting, a unified framework and two-stage training paradigm that integrates scene composition, multi-view subject consistency, and motion control. Our approach leverages a dual-condition motion module driven by 3D tracking points for background scenes and downsampled RGB cues for foreground subjects. To ensure a balance between controllability and visual realism, we further propose an inference ControlNet scale schedule. Tri-Prompting supports novel workflows, including 3D-aware subject insertion into any scenes and manipulation of existing subjects in an image. Experimental results demonstrate that Tri-Prompting significantly outperforms specialized baselines such as Phantom and DaS in multi-view subject identity, 3D consistency, and motion accuracy.
