Search papers, labs, and topics across Lattice.
The paper introduces SkillsCrafter, a robotic manipulation framework designed for continual learning of multiple language-conditioned manipulation skills while mitigating catastrophic forgetting. SkillsCrafter employs a Manipulation Skills Adaptation module to retain old skills knowledge and inherit shared knowledge for new skill learning. It also uses Skills Specialization Aggregation, which computes inter-skill similarity in a learned semantic subspace derived from SVD of diverse skill instructions, to aggregate previously learned knowledge for new or unknown skills.
SkillsCrafter enables robots to learn new manipulation skills without forgetting old ones by distilling knowledge into a shared semantic space, paving the way for more adaptable and robust robotic systems.
Traditional language-conditioned manipulation agent sequential adaptation to new manipulation skills leads to catastrophic forgetting of old skills, limiting dynamic scene practical deployment. In this paper, we propose SkillsCrafter, a novel robotic manipulation framework designed to continually learn multiple skills while reducing catastrophic forgetting of old skills. Specifically, we propose a Manipulation Skills Adaptation to retain the old skills knowledge while inheriting the shared knowledge between new and old skills to facilitate learning of new skills. Meanwhile, we perform the singular value decomposition on the diverse skill instructions to obtain common skill semantic subspace projection matrices, thereby recording the essential semantic space of skills. To achieve forget-less and generalization manipulation, we propose a Skills Specialization Aggregation to compute inter-skills similarity in skill semantic subspaces, achieving aggregation of the previously learned skill knowledge for any new or unknown skill. Extensive experiments demonstrate the effectiveness and superiority of our proposed SkillsCrafter.
