Search papers, labs, and topics across Lattice.
This paper introduces the Continual IVON (CoVON) optimizer, which integrates fast and slow adaptation mechanisms into the Variational Continual Learning (VCL) framework by merging past posteriors to stabilize knowledge retention. The approach addresses the limitations of existing optimizers that struggle with continual adaptation, demonstrating that CoVON outperforms traditional VCL optimizers and weight-regularization strategies in various learning scenarios. Key results indicate that CoVON enhances performance in domain-incremental learning, continual pre-training, and fine-tuning of large language models, showcasing its practical applicability in deep learning tasks.
Merging past knowledge with fast adaptation in the CoVON optimizer leads to superior performance in continual learning tasks, outperforming traditional methods.
Continual learning remains a major challenge for modern deep networks, partly because commonly used optimizers lack inherent mechanisms for continual adaptation. One such natural mechanism is fast and slow adaptation to balance stability and plasticity. This mechanism has deep roots in neuroscience and biology, but there is no consensus on how to best incorporate it in commonly used optimizers. Here, we show that this can be easily done via the VCL framework, where past posteriors are used as priors in the future. Our key idea is to incorporate slow adaptation via merging of past posteriors to slow down the drift in the knowledge as learning progresses. The merged posterior is then used as the prior in the VCL update to implement the fast-weight updates. These steps can be seamlessly implemented in the IVON optimizer, whose form and costs are nearly identical to that of Adam. We call this new optimizer the Continual IVON (CoVON) optimizer and show that it not only consistently improves over existing VCL optimizers, but also performs better than other weight-regularization strategies across domain-incremental learning, continual pre-training, and fine-tuning of large language models.
