Search papers, labs, and topics across Lattice.
This paper addresses catastrophic forgetting in continual learning (CL) by improving replay buffer selection, particularly under memory constraints. They introduce Multiple Embedding Replay Selection (MERS), a graph-based approach that integrates both supervised and self-supervised embeddings to select samples for the replay buffer. Experiments on CIFAR-100 and TinyImageNet demonstrate that MERS consistently outperforms state-of-the-art selection strategies, especially in low-memory regimes, without increasing model parameters or replay volume.
Self-supervised embeddings, often ignored in continual learning replay buffers, unlock surprisingly large gains in catastrophic forgetting, especially when memory is tight.
Catastrophic forgetting remains a key challenge in Continual Learning (CL). In replay-based CL with severe memory constraints, performance critically depends on the sample selection strategy for the replay buffer. Most existing approaches construct memory buffers using embeddings learned under supervised objectives. However, class-agnostic, self-supervised representations often encode rich, class-relevant semantics that are overlooked. We propose a new method, Multiple Embedding Replay Selection, MERS, which replaces the buffer selection module with a graph-based approach that integrates both supervised and self-supervised embeddings. Empirical results show consistent improvements over SOTA selection strategies across a range of continual learning algorithms, with particularly strong gains in low-memory regimes. On CIFAR-100 and TinyImageNet, MERS outperforms single-embedding baselines without adding model parameters or increasing replay volume, making it a practical, drop-in enhancement for replay-based continual learning.
