Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The authors present FlexMoGen, a latent diffusion framework that conditions human motion generation simultaneously on natural language descriptions and raw reference style clips. While text prompts capture high-level semantics effectively, they fail to specify fine-grained kinematics like joint articulation and timing, which existing style-conditioned models typically constrain to rigid, discrete category labels. By jointly training an unsupervised variational style encoder with a text-to-motion backbone, the model enables long-horizon, time-varying, and multi-style animation synthesis with superior Pareto optimality between text-content alignment and style fidelity.
Text defines the action, but reference clips dictate the nuance: pairing latent diffusion with an unsupervised style encoder unlocks long-horizon, time-varying motion synthesis from arbitrary kinematic examples without discrete style labels.
We introduce FlexMoGen, a novel framework for flexible human motion synthesis conditioned on both natural language descriptions and motion style references. Text prompts are effective at defining semantic content, but they are often limited in capturing fine-grained style details such as timing, limb articulation, and expressive dynamics. A style example clip supplements the text by conveying these nuanced motion characteristics directly, enabling the model to preserve high-level intent while reproducing the desired stylistic traits. Given a text prompt and a style example clip, FlexMoGen generates high-quality motions that preserve semantic content while faithfully reflecting the target style, offering users greater control over the animation generation process. Unlike prior methods that rely on discrete style labels and do not generalize to long or multi-style generation, FlexMoGen learns a variational style encoder without style supervision and supports long, time-varying, multi-style synthesis. Our framework jointly pre-trains the style encoder and a text-to-motion latent diffusion model within a unified architecture, modulating motion style through a lightweight adaptation module. It integrates an efficient relative positional encoding scheme and is trained on both stylized and non-stylized datasets, enabling strong generalization to unseen text-style combinations. Experiments show that FlexMoGen achieves the best balance between content fidelity and style reflection.