Search papers, labs, and topics across Lattice.
The paper introduces Rhythm, a framework for enabling real-world dual-humanoid interaction by addressing kinematic mismatches and complex contact dynamics. Rhythm uses Interaction-Aware Motion Retargeting (IAMR) to generate feasible interaction references from human data, Interaction-Guided Reinforcement Learning (IGRL) with graph-based rewards to learn coupled dynamics, and a deployment system for sim-to-real transfer. Experiments on Unitree G1 robots demonstrate successful transfer of complex interactive behaviors like hugging and dancing.
Watch two robots hug and dance: Rhythm enables robust, physically plausible interactions between dual humanoids, bridging the gap from simulation to real-world deployment.
Realizing interactive whole-body control for multi-humanoid systems is critical for unlocking complex collaborative capabilities in shared environments. Although recent advancements have significantly enhanced the agility of individual robots, bridging the gap to physically coupled multi-humanoid interaction remains challenging, primarily due to severe kinematic mismatches and complex contact dynamics. To address this, we introduce Rhythm, the first unified framework enabling real-world deployment of dual-humanoid systems for complex, physically plausible interactions. Our framework integrates three core components: (1) an Interaction-Aware Motion Retargeting (IAMR) module that generates feasible humanoid interaction references from human data; (2) an Interaction-Guided Reinforcement Learning (IGRL) policy that masters coupled dynamics via graph-based rewards; and (3) a real-world deployment system that enables robust transfer of dual-humanoid interaction. Extensive experiments on physical Unitree G1 robots demonstrate that our framework achieves robust interactive whole-body control, successfully transferring diverse behaviors such as hugging and dancing from simulation to reality.
