Search papers, labs, and topics across Lattice.
This paper introduces DeMoDiff, a spatiotemporally decoupled framework for human motion generation that enhances representation extraction and part-level controllability by encoding individual body joints instead of whole-body motion. By integrating spatial-temporal masking and attention mechanisms into an autoregressive diffusion generator, the model achieves superior generative capabilities and fine-grained editability. Extensive evaluations on the HumanML3D and KIT-ML datasets reveal that DeMoDiff outperforms existing methods in both reconstruction quality and motion generation diversity.
Achieving state-of-the-art motion generation, DeMoDiff allows for unprecedented control over individual body parts while maintaining high-quality outputs.
Text-driven human motion synthesis has made substantial development with two core modules of motion representation and generative architecture. For representation, Vector Quantization (VQ)-based methods compress motion data into discrete tokens while latent-based models operate directly in continuous space. However, both of these representations exhibit significant limitations. VQ-based methods suffer from inherent information loss, which compromises the quality, diversity, and generalization of generated motions, while continuous representation on holistic whole-body motion hinders part-level flexibility. For architecture, diffusion and autoregressive diffusion models have demonstrated their superiority, yet the fine-grained controllability over individual body parts is also limited. Thus, we propose a unified spatiotemporally decoupled framework named DeMoDiff, which jointly redesigns representation and architecture. To enhance representation extraction capabilities and offer greater part-level controllability, we present a spatial-temporal VAE that encodes each body joint rather than compressing the whole-body motion into a single latent space. Then, we incorporate spatial-temporal masking and attention mechanisms into an autoregressive diffusion generator, achieving both generative capability and controllable editability. Extensive experiments on the HumanML3D and KIT-ML datasets demonstrate that our model achieves state-of-the-art reconstruction performance and compelling motion generation results. Moreover, our framework demonstrates strong temporal and spatial editing capabilities, further validating its effectiveness. Our project page: https://rex0191.github.io/DeMoDiff/
