Search papers, labs, and topics across Lattice.
This paper introduces BooST, a two-stage framework that effectively bridges semantic intent and motion dynamics for skill transfer in robotic learning. By utilizing a cross-modal VQ-VAE to create a unified skill representation, BooST enhances sample efficiency and generalization across tasks while ensuring robustness to visual and dynamic perturbations. Experimental results show that BooST outperforms existing methods in few-shot adaptation and cross-domain skill transfer, making it suitable for practical deployment in real-world scenarios.
Bridging semantics and motion dynamics, BooST achieves superior skill transfer and adaptation in robotics, outperforming traditional methods.
Skill abstraction---the process of learning reusable and temporally extended behaviors---has emerged as a key paradigm for improving sample efficiency and generalization in robot learning. For efficient skill transfer to real robots, learned skills must generalize across tasks and domains, remain robust to visual and dynamic perturbations, and be efficient enough for practical deployment. However, existing methods typically satisfy only a subset of these properties, as they capture either high-level semantic intent (what) or low-level motion dynamics (how). This incomplete skill transfer yields weak priors for policy learning, thereby demanding substantial in-domain data for downstream adaptation. To address these challenges, we introduce BooST, a two-stage framework that explicitly bridges semantics and motions to satisfy all three desiderata. BooST first leverages a cross-modal VQ-VAE to capture both semantic intent and motion dynamics, yielding a unified skill representation. It then distills this representation into a lightweight policy for efficient downstream adaptation to new tasks. Extensive experiments across simulation and real-robot settings demonstrate that BooST achieves superior few-shot adaptation, cross-domain skill transfer, and robustness to dynamic visual distractors, while maintaining a lightweight yet expressive design suitable for real-world deployment.
