Search papers, labs, and topics across Lattice.
This paper introduces the concept of Memory Anchors, which are specific experiences that significantly aid in preserving learned behaviors during continual robot learning. By identifying and utilizing these anchors in regions where new-task observations overlap with old-task representations, the authors demonstrate a marked reduction in catastrophic forgetting. Their findings reveal that excluding just 10% of these anchors can lead to over a 4.5x increase in forgetting, while incorporating them can reduce high-conflict task forgetting by 63%, facilitating effective continual learning on real robots.
Excluding just 10% of critical Memory Anchors can lead to a staggering 4.5x increase in catastrophic forgetting in robots.
Robot policies deployed in the wild should have the capability to continually learn new tasks without forgetting existing behaviors. A common approach to combat such catastrophic forgetting is to train on new task data with a replay buffer of previously learned task data. Although this buffer is commonly sampled randomly from all prior experiences, we show that a small set of these experiences contributes greatly in anchoring past performance. We call these experiences Memory Anchors. We identify Memory Anchors in regions where representations of new-task observations collapse onto those of old-task observations even though the tasks require conflicting actions, like when a familiar object must be manipulated in a new way. Rehearsing old data in this region plays a key role in preventing destructive overwriting of past task knowledge, serving as this critical Memory Anchor role. Excluding only 10% Memory Anchors before sampling the buffer leads to more than a 4.5x increase in catastrophic forgetting on the LIBERO benchmark suites. Conversely, enriching the replay buffer with Memory Anchors can decrease high-conflict task forgetting by 63% and enables successful continual learning of two task sequences on a real robot. Videos and additional visualizations can be found at https://robot-adaptation.github.io/MemoryAnchors
