Search papers, labs, and topics across Lattice.
The paper introduces SparTa, a method for learning sparse graphical task models for long-horizon manipulation tasks from a small number of demonstrations by inferring the desired object state changes rather than directly learning actions. SparTa uses a demonstration segmentation and pooling approach to extract manipulation graphs representing object relationships and estimates distributions over object states across task phases, capturing complete object interactions over long temporal windows. Experiments demonstrate that SparTa achieves high demonstration segmentation accuracy, benefits from learning from multiple demonstrations, and enables reliable task execution in simulation and on a real robot.
Skip directly learning actions and instead infer the "what" of robotic manipulation tasks by extracting sparse graphical task models from just a few demonstrations.
Learning long-horizon manipulation tasks efficiently is a central challenge in robot learning from demonstration. Unlike recent endeavors that focus on directly learning the task in the action domain, we focus on inferring what the robot should achieve in the task, rather than how to do so. To this end, we represent evolving scene states using a series of graphical object relationships. We propose a demonstration segmentation and pooling approach that extracts a series of manipulation graphs and estimates distributions over object states across task phases. In contrast to prior graph-based methods that capture only partial interactions or short temporal windows, our approach captures complete object interactions spanning from the onset of control to the end of the manipulation. To improve robustness when learning from multiple demonstrations, we additionally perform object matching using pre-trained visual features. In extensive experiments, we evaluate our method's demonstration segmentation accuracy and the utility of learning from multiple demonstrations for finding a desired minimal task model. Finally, we deploy the fitted models both in simulation and on a real robot, demonstrating that the resulting task representations support reliable execution across environments.
