Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
SkillWeave addresses the challenge of collecting heterogeneous demonstrations for long-horizon dexterous manipulation by combining teleoperation for coarse movements with kinesthetic teaching for precise contact-rich skills. The framework employs an object-mask-conditioned diffusion policy to handle visual mismatches from kinesthetic teaching and successor-aware terminal steering to mitigate distribution shifts during policy handoffs. This approach achieves 27% average end-to-end success on three real-world tasks, with kinesthetic policies improving sub-task success to 65% and handoffs reaching 87% composition efficiency.
A single framework successfully integrates teleoperation and kinesthetic teaching for long-horizon manipulation, achieving 27% end-to-end success by novelly handling visual mismatches and steering policy handoffs.
Dexterous manipulation requires both large-scale task progression and precise contact-rich interaction, making it challenging to collect demonstrations that effectively support both regimes. We present SkillWeave, a heterogeneous demonstration framework for long-horizon dexterous manipulation that combines teleoperation for coarse reaching and transport with kinesthetic teaching for precise, contact-rich skills. To address the visual mismatch introduced by the demonstrator's presence during kinesthetic data collection, we propose an object-mask-conditioned diffusion policy that uses offline object segmentation for training supervision and a lightweight learned mask predictor at deployment, avoiding online segmentation and image inpainting. To mitigate distribution shift between independently trained sub-task policies, we introduce successor-aware terminal steering, which selects among actions sampled from the predecessor policy to guide the system toward states supported by the successor's demonstrated initial-state distribution. Across three real-world long-horizon tasks, SkillWeave achieves 27% average end-to-end success. Mask-conditioned kinesthetic policies improve dexterous sub-task success to an average of 65%, while successor-aware handoffs achieve an average composition efficiency of 87%. These results show that matching demonstration modality to interaction regime, explicitly addressing kinesthetic visual mismatch, and steering policy handoffs toward successor-supported states substantially improves long-horizon dexterous manipulation. Videos and code are available at skillweave-authors.github.io .