Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
DexRoam establishes an end-to-end framework to train mobile bimanual dexterous robots using egocentric human demonstrations captured solely via a consumer VR headset and head-mounted stereo camera without external tracking rigs. To prevent the degradation of continuous locomotion and manipulation dynamics common in prior transfer methods, the system maps human motion into robot actions using a three-stage pipeline spanning embodiment, action-semantic, and temporal alignment. Co-training on these aligned demonstrations nearly doubles real-world task success across leading VLA backbones (from 29% to 56% on GR00T N1.7 and 32% to 57% on pi0.5) while cutting the required physical robot demonstrations by half.
Consumer VR headsets can slash physical robot data collection requirements in half while nearly doubling real-world success on complex, whole-body mobile dexterous manipulation tasks.
Mobile bimanual dexterous manipulation requires continuous coordination of locomotion, whole-body motion, and finger-level dexterity within a single trajectory, creating a severe robot demonstration bottleneck. Egocentric human demonstrations offer a scalable alternative, but prior approaches ease the transfer by simplifying human motion, discarding exactly the fine-grained, coupled structure such tasks depend on. We present DexRoam, a complete system for learning mobile bimanual dexterous manipulation from human demonstrations, in which whole-body motion remains continuous and coupled throughout the human-to-robot transfer process. To enable scalable collection of whole-body human manipulation demonstrations, we develop a tracker-free capture system using only a consumer VR headset and a head-mounted stereo camera, without external cameras or motion trackers. We then perform three explicit alignment stages---embodiment, action-semantic, and temporal---to map captured motion into the robot action space, preserving fine-grained whole-body motion and allowing human and robot demonstrations to be jointly learned by standard VLA policies. Real-world experiments with different VLA backbones show that human demonstrations consistently improve policy learning across training paradigms, raising average success from 29% to 56% on GR00T N1.7 and from 32% to 57% on pi0.5, while matching robot-only training with half the robot demonstrations. Ablations confirm that each alignment stage is necessary. These results highlight the potential of human demonstrations for scalable whole-body mobile manipulation with preserved fine-grained motion structure.