Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
This paper introduces a continual learning framework for single-view 6-DoF grasp synthesis in cluttered scenes, addressing performance degradation on unseen objects. The method adapts by updating grasp scores based on past outcomes and incorporating user demonstrations as candidate grasps within a learned embedding space, avoiding full model finetuning. Evaluated in simulation and extensive real-world trials, the approach matches baseline performance initially, significantly improves on novel objects, and demonstrates robust long-horizon learning with minimal forgetting, achieving over 90% success on challenging categories after limited online adaptation.
Robots can now learn to grasp novel objects in real-time, achieving over 90% success after just 50 attempts without retraining.
Most current grasp synthesis systems are trained offline and remain fixed during deployment. While this works well when deployment conditions resemble the training data, performance can degrade when robots encounter conditions they have not seen before, such as unfamiliar objects. In this work, we present a continual-learning framework for single-view 6-DoF grasp synthesis for a parallel-jaw gripper in cluttered scenes. Rather than finetuning a large parametric model, our method adapts through memory in a learned embedding space: grasp outcomes update future grasp scores, while optional user demonstrations are recalled and transferred to new scenes as additional candidate grasps. We evaluate our method in simulation and in extensive real-world experiments comprising over 1500 grasp trials. We show that our method matches the performance of existing 6-DoF grasping baselines even before adaptation, improves online on unseen objects from categories absent or underrepresented during training, and supports long-horizon continual learning with limited forgetting. In real-world experiments, our method reaches over 90\% success rates on several challenging object categories after only 50 online grasp attempts. Videos and code at https://giuschio.github.io/cl_grasping/.