Search papers, labs, and topics across Lattice.
This paper presents a novel approach to video motion transfer that transcends traditional fixed structural correspondences, enabling the animation of target objects with dynamics from reference videos even when they differ significantly in morphology. The proposed two-stage framework first learns abstract motion representations that capture transferable dynamics across diverse categories and then utilizes these representations for direct generation without the need for explicit motion extraction. Experimental results show that this method achieves state-of-the-art performance in terms of motion fidelity and target preservation, supported by the introduction of the OpenVMT-Dataset and OpenVMT-Bench for comprehensive evaluation.
Motion transfer can now succeed across vastly different object categories, achieving state-of-the-art fidelity without relying on fixed structural correspondences.
Video motion transfer aims to animate a target object using dynamics from a reference video. Existing formulations largely rely on fixed structural correspondence, which becomes ill-defined when reference and target objects differ substantially in morphology, articulation, or deformation mechanisms. We introduce Motion Beyond Morphology, a perspective that seeks to transfer motion beyond fixed structural correspondence, by preserving dynamics that remain meaningful across different target morphologies. To realize this, we propose a two-stage framework. Stage~I learns complementary multi-granularity abstract motion views and uses them to bootstrap cross-category video pairs that preserve transferable dynamics across diverse morphologies. Stage~II internalizes this supervision into direct reference-video-conditioned generation, removing the need for explicit motion extraction at inference. We further introduce OpenVMT-Dataset and OpenVMT-Bench for training and evaluating image- and text-conditioned motion transfer across Same, Near, and Far category gaps. Extensive experiments demonstrate state-of-the-art motion fidelity and target preservation. Project page: https://miniz233.github.io/MotionBeyondMorphology/
