Search papers, labs, and topics across Lattice.
This paper critiques the traditional focus on knowledge retention in Continual Learning (CL), arguing that prioritizing retention can hinder adaptation in non-stationary environments. By redefining CL as an online optimization problem and introducing Transfer Efficiency as a measure of the balance between conflicting past experiences and the cost of learning new tasks, the authors derive a Critical Task Duration that indicates when historical knowledge becomes a liability. Their proposed Predictive Continual Learning algorithms, which dynamically optimize for future performance, demonstrate superior performance on benchmarks compared to both Joint-Task Learning and Independent-Task Learning approaches.
Retaining past knowledge can actually impede real-time adaptation in dynamic environments, leading to a new framework for optimizing continual learning.
The Continual Learning (CL) literature has long been driven by the goal of mitigating catastrophic forgetting. This objective rests on a pervasive, often unstated assumption: that a lifelong learner should approximate the Joint-Task Learning (JTL) solution and retain all previously acquired knowledge. We challenge this retention-centered premise, arguing that in non-stationary environments prioritizing retention can impede real-time adaptation. Shifting the focus to the Average Lifelong Error (ALE), we formalize CL as an online optimization problem governed by the interaction between environmental and learning dynamics. We introduce Transfer Efficiency as a quantitative measure of the tension between Instability, the bias inherited from conflicting past experience, and Transient Error, the optimization cost of learning new tasks from scratch. Under mild convergence conditions, holding across linear and neural network models, this decomposition yields a Critical Task Duration: a closed-form threshold beyond which historical knowledge transitions from a warm-start advantage to an optimization liability whenever retention induces a positive stationary bias. We validate these theoretical predictions on continual image classification and reinforcement learning benchmarks. Finally, by connecting continual learning to the online learning framework of predictable sequences, we show that JTL is only one instance of a broader family of objectives, and we propose a new general class of continual learning algorithms, which we call Predictive Continual Learning. Predictive CL algorithms optimize expected future performance under an explicit, dynamically updated model of future tasks. As a proof of concept, we analyze a Window algorithm that interpolates between JTL and Independent-Task Learning (ITL), outperforming both under controlled distributional drift.
ETH
Mila