Search papers, labs, and topics across Lattice.
This paper introduces DR-LfD, a novel framework that integrates visuomotor policies into task and motion planning (TAMP) to enhance robotic manipulation. By decomposing human demonstrations into atomic skills and modeling them in a TAMP-compatible format, DR-LfD significantly reduces the data requirements for training while improving spatial generalization and multi-stage operation capabilities. Experimental results show that DR-LfD outperforms traditional methods in complex, contact-rich environments, demonstrating its effectiveness in real-world and simulated scenarios.
Reducing the demonstration burden for robotic manipulation tasks by leveraging atomic skills learned from human demonstrations could revolutionize how robots learn and execute complex actions.
Successfully automating dexterous, long-horizon robotic manipulation requires frameworks capable of both high-level reasoning and fine-grained execution. Traditional task and motion planning (TAMP), while excellent at symbolic planning, is often brittle in contact-rich operations. Simultaneously, imitation learning (IL), while effective in manipulation tasks with visual feedback, is limited by its low capability in spatial generalization and multi-stage operation. To reconcile their complementary strengths and limitations, we propose DR-LfD (Decomposed and Reorganized Skills Learned from Demonstrations), a framework that seamlessly integrates visuomotor policies into a TAMP-gated decision-making system. Based on contact relationships, DR-LfD decomposes human demonstrations into atomic skills, which are reproduced as visuomotor policies or object-centric primitives. The initiation, termination, and constraints of the visuomotor policies are carefully modeled and implemented in a TAMP-compatible form, enabling reorganization of skills learned from different sources. DR-LfD transforms the learning problem from one requiring exponential demonstration data over possible skill sequences to one whose demonstration burden scales with the number of distinct skill types, with limited data for each skill. Through comprehensive real-world and simulation benchmarking across diverse scenarios, we demonstrate the strong performance of DR-LfD on tasks involving multiple steps, unseen setups, and physical constraints. Project website: https://dr-lfd.github.io/DR-LfD-website.
