Search papers, labs, and topics across Lattice.
InterReal, a physics-based imitation learning framework, is introduced to enable humanoid robots to learn and execute real-world human-object interaction (HOI) skills. The framework incorporates a HOI motion data augmentation scheme with contact constraints to improve policy robustness, and an automatic reward learner guided by tracking error metrics for effective reward shaping. Experiments in simulation and on a Unitree G1 robot demonstrate that InterReal outperforms baselines in tracking accuracy and task success for box-picking and box-pushing tasks.
Robots can now learn fine-grained manipulation skills directly from human demonstrations, thanks to a new imitation learning framework that automatically figures out the right rewards and handles noisy real-world data.
Interaction is one of the core abilities of humanoid robots. However, most existing frameworks focus on non-interactive whole-body control, which limits their practical applicability. In this work, we develop InterReal, a unified physics-based imitation learning framework for Real-world human-object Interaction (HOI) control. InterReal enables humanoid robots to track HOI reference motions, facilitating the learning of fine-grained interactive skills and their deployment in real-world settings. Within this framework, we first introduce a HOI motion data augmentation scheme with hand-object contact constraints, and utilize the augmented motions to improve policy stability under object perturbations. Second, we propose an automatic reward learner to address the challenge of large-scale reward shaping. A meta-policy guided by critical tracking error metrics explores and allocates reward signals to the low-level reinforcement learning objective, which enables more effective learning of interactive policies. Experiments on HOI tasks of box-picking and box-pushing demonstrate that InterReal achieves the best tracking accuracy and the highest task success rate compared to recent baselines. Furthermore, we validate the framework on the real-world robot Unitree G1, which demonstrates its practical effectiveness and robustness beyond simulation.
