Search papers, labs, and topics across Lattice.
This paper introduces a text-conditioned diffusion model designed to synthesize realistic full-body human interactions with articulated objects, addressing the limitations of existing models that either focus on simple tasks or hand-only manipulation. The proposed approach integrates an object-centric representation, a mixed-domain training strategy, and a contact-based augmentation scheme to enhance generalization across diverse object configurations. Experimental results show that this method significantly outperforms current state-of-the-art techniques in generating coordinated locomotion and manipulation of articulated objects.
Surpassing current methods, this model achieves unprecedented generalization in full-body interactions with articulated objects, even in unseen configurations.
Synthesizing realistic full-body human interactions with articulated objects is a fundamental challenge for embodied AI and graphics, with applications in robotics training and virtual agents. Existing models remain limited: some focus on simple activities with static objects, while others restrict attention to hand-only manipulation. This leaves open the problem of generating coordinated full-body motion that approaches, manipulates, and moves articulated objects in a realistic and generalizable way. The key difficulty lies in reasoning jointly about locomotion, fine-grained contact, and object articulation. Models must capture subtle hand-object correspondences that transfer across object geometries, while also producing seamless transitions from navigation to manipulation. At the same time, the scarcity of large-scale paired motion-scene data makes it difficult to generalize across diverse object positions and shapes. We introduce a text-conditioned diffusion model that addresses these challenges through three core ideas: an object-centric representation that unifies hand-object contact with object surfaces, a mixed-domain training strategy that balances locomotion and interaction, and a contact-based augmentation scheme that expands training diversity. Through experiments, our method demonstrated strong generalization to unseen object configurations, surpassing current state-of-the-art methods.
