Search papers, labs, and topics across Lattice.
This paper introduces ACT, a trajectory-conditioned framework that enhances skeletal animation by utilizing 3D point trajectories from monocular video as motion guidance, addressing the limitations of existing methods that often decouple 3D asset generation from motion synthesis. By employing a Routed Trajectory Injector with innovative routing techniques, ACT achieves precise skeleton-to-mesh correspondences and improved temporal alignment, resulting in high-fidelity animations with robust motion consistency. Extensive experiments show that ACT significantly outperforms traditional approaches, offering a more efficient and controllable solution for 4D animation generation.
ACT revolutionizes skeletal animation by achieving unprecedented fidelity and temporal consistency through innovative trajectory conditioning.
4D generation aims to animate 3D objects with realistic motion, holding great promise for applications. Existing methods typically decouple 3D asset generation from motion synthesis: acquire a 3D asset, prepare a structural representation like mesh and Gaussians, and synthesize motion from text or video control signals. However, dense mesh and Gaussian representations incur high computational costs and are prone to temporal artifacts, limiting animation quality and duration to only short clips. Meanwhile, text lacks fine-grained spatial and temporal details such as timing and coordination, while video entangles motion with appearance and background. Together, these limitations result in 4D animations that suffer from poor temporal consistency, wrong identification, and limited controllability. We address these issues with \texttt{ACT}, a trajectory-conditioned framework for topology-general skeletal animation. ACT uses skeletons as a compact structured and compute-efficient representation and 3D point trajectories from monocular video as explicit motion guidance which provide detailed motion patterns without appearance entanglement. At the core of ACT is a Routed Trajectory Injector, which achieves accurate and robust trajectory-to-joint transfer through three complementary designs: prior-guided hard routing establishes precise skeleton-to-mesh correspondences, global routing enables holistic joint-track interaction for full-body motion awareness, and local windowed cross-attention enforces fine-grained temporal alignment, improving micro-timing and reducing motion misalignment across varying motion rates. Extensive experiments demonstrate that \texttt{ACT} significantly outperforms existing methods in fidelity and temporal consistency.
