Search papers, labs, and topics across Lattice.
This paper investigates the integration of handheld data collection and teleoperated demonstrations for improving robot manipulation in contact-rich environments. By analyzing the limitations of both methods, the authors propose a novel approach called Bi-modal Routing for Imitation Data via Gated Experts (BRIDGE), which selectively utilizes teleoperated data only in critical task phases where handheld data fails. The results show that this hybrid strategy can enhance success rates by up to 36.7% compared to using handheld data alone across multiple tasks, addressing the challenges of dynamic feasibility in contact-sensitive scenarios.
Combining handheld and teleoperated data can boost robot manipulation success rates by over 36%, but only if done strategically.
Handheld data collection systems, such as the Universal Manipulation Interface (UMI), enable scalable data collection across diverse environments but only capture observed actions rather than the desired actions executed by a robot controller. In contrast, teleoperation captures desired actions directly, but is prohibitively time-consuming to collect. We revisit this trade-off through the lens of action validity across task phases. We observe that handheld trajectories provide valid supervision in tolerant, free-space phases, but lack dynamic feasibility in contact-sensitive phases, where tracking observed trajectories at high stiffness produces large, unsafe contact forces. We study the interaction between these two supervision types for contact-rich manipulation and find that training policies that combine handheld data with a small number of targeted teleoperated demonstrations provide an efficient hybrid strategy. Specifically, rather than teleoperating the entire task, we only collect partial teleoperated demonstrations for task segments where base handheld policies fail. However, naively mixing handheld and teleoperated phase-specific data yields worse performance than training on handheld data alone. To address this mismatch between observed and desired supervision, we propose Bi-modal Routing for Imitation Data via Gated Experts (BRIDGE), a mixture of diffusion policy experts that routes between specialist task phase heads conditioned on the current robot state. Notably, our approach enables task-phase specific use of desired actions during contact sensitive segments and improves success rates over handheld-only baselines by up to 36.7% across three contact-rich manipulation tasks.
