Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study reframes catastrophic forgetting in continual learning as a loss of accessibility to preserved knowledge rather than complete erasure, introducing a three-level framework that distinguishes between knowledge storage, representation, and accessibility. Through experiments on sequential CIFAR-100 classification using ResNet-18, the authors demonstrate that while task accuracy can collapse dramatically, a significant portion of representational information remains retrievable, with linear probing retaining about 76% of the original information. Notably, retraining only the final classifier can recover 75.7% of task performance, indicating that critical task information is preserved in the network's early and intermediate layers despite apparent forgetting.
Catastrophic forgetting may not erase knowledge but instead hide it, with up to 76% of information remaining accessible through clever retraining strategies.
Catastrophic forgetting is commonly interpreted as the irreversible erasure of previously acquired knowledge during sequential learning. In this work, we investigate an alternative perspective: that forgetting may arise not from complete destruction of task representations but from a loss of accessibility to preserved information. We introduce a three-level framework separating knowledge storage, representation, and accessibility, and evaluate each component through a series of continual-learning experiments on sequential CIFAR-100 classification using ResNet-18. Our analysis combines checkpoint persistence, linear probing, representation geometry, classifier-reset recovery, and layer-wise recoverability experiments. We observe complete behavioral forgetting of earlier tasks, with task accuracy collapsing from 54.8% to 0%, while linear probe performance retains approximately 76% of the original representational information. Furthermore, retraining only the final classifier restores 75.7% of the original task performance without modifying the backbone network. Layer-wise analysis reveals that early and intermediate layers preserve highly recoverable task information despite severe degradation at later stages. Projection-energy and principal-angle analyses indicate that retained knowledge persists as distributed high-dimensional representations rather than through preservation of a small dominant subspace. These findings suggest that catastrophic forgetting is better characterized as an accessibility failure than complete representational erasure, and that substantial task-relevant information remains embedded within neural representations even after functional forgetting has occurred.