Search papers, labs, and topics across Lattice.
The paper introduces HoMMI, a framework for learning whole-body mobile manipulation policies from robot-free human demonstrations using egocentric sensing. To address the human-to-robot embodiment gap, they employ a cross-embodiment hand-eye policy design with embodiment-agnostic visual representations and a whole-body controller. The framework enables learning of long-horizon mobile manipulation tasks requiring bimanual coordination, navigation, and active perception.
Skip the expensive robots: HoMMI lets you train whole-body mobile manipulation policies just from human demos, thanks to a clever cross-embodiment design.
We present Whole-Body Mobile Manipulation Interface (HoMMI), a data collection and policy learning framework that learns whole-body mobile manipulation directly from robot-free human demonstrations. We augment UMI interfaces with egocentric sensing to capture the global context required for mobile manipulation, enabling portable, robot-free, and scalable data collection. However, naively incorporating egocentric sensing introduces a larger human-to-robot embodiment gap in both observation and action spaces, making policy transfer difficult. We explicitly bridge this gap with a cross-embodiment hand-eye policy design, including an embodiment agnostic visual representation; a relaxed head action representation; and a whole-body controller that realizes hand-eye trajectories through coordinated whole-body motion under robot-specific physical constraints. Together, these enable long-horizon mobile manipulation tasks requiring bimanual and whole-body coordination, navigation, and active perception. Results are best viewed on: https://hommi-robot.github.io
