Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
4Director addresses the limitations of coarse object control in video generation by introducing a novel approach that conditions a video world model on explicit rigid 3D geometry. By reconstructing objects as canonical meshes and applying prescribed rigid transformations per frame, the model achieves precise camera and object motion control, overcoming ambiguities inherent in image-plane cues. The system renders a depth video and employs a Motion Adapter to synthesize view-consistent appearance and dynamics, achieving superior visual quality and control over existing methods on the new RealCOD-Rigid dataset.
Achieve precise, intuitive 3D control over video generation by moving rigid object meshes, eliminating depth ambiguity and viewpoint inconsistencies.
Precise control over camera and object motion is essential for professional video production. Existing methods control objects only coarsely, through image-plane cues that are ambiguous in depth and rotation or through 3D tracks and blobs that lack complete geometry and lose consistency across viewpoint changes. We introduce 4Director, a video world model conditioned on an explicit 4D scene representation: each object is reconstructed once from the input image as a canonical mesh and moved by one prescribed rigid transformation per frame. This representation provides an intuitive 3D control interface and prevents unobserved geometry from being regenerated independently in every frame. We render the controlled scene as a depth video and introduce a Motion Adapter that transforms this geometric scaffold into video while synthesizing view-consistent appearance, illumination, and non-rigid dynamics. For training, we construct RealCOD-Rigid, a new dataset of 20,774 clips annotated with rigid 3D scenes by our automatic pipeline. We further introduce Identity-Gated IoU (IG-IoU), which jointly evaluates adherence to prescribed object motion and preservation of object identity. Experiments demonstrate that 4Director consistently outperforms prior methods in visual quality and in camera and object control.