Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study investigates the geometric principles underlying recoverability in continual learning, specifically addressing catastrophic forgetting through the lens of the Accessibility Collapse framework. By analyzing recoverability, representational drift, and recovery complexity in a sequentially trained ResNet-18 on Split CIFAR-100, the authors introduce Recovery Subspace Dimensionality (k_t), which remains stable at a mean of 8.0 despite significant representational drift. The findings reveal that forgotten knowledge is still compactly decodable, suggesting that the challenges of catastrophic forgetting stem more from accessibility issues than from the actual loss of information.
Recovery dimensionality remains stable even as representational drift occurs, challenging the conventional view of catastrophic forgetting as mere information loss.
Catastrophic forgetting is often viewed as the destruction of previously learned knowledge during sequential learning. Building on the Accessibility Collapse framework, we investigate the geometric structure of recoverability in continual learning. Using Split CIFAR-100 and a sequentially trained ResNet-18, we analyze recoverability, representational drift, and recovery complexity across ten tasks. We introduce Recovery Subspace Dimensionality (k_t), a measure of the minimum number of singular directions required to preserve 90 percent of full probe performance. Contrary to our Recoverability Diffusion hypothesis, recovery dimensionality remains stable throughout training (mean k_t = 8.0) despite substantial representational drift. Principal-angle drift strongly predicts recoverability (r = -0.862), and a simple geometric model explains 82.2 percent of recoverability variance. These findings support the Stable Recovery Manifold hypothesis, suggesting that forgotten knowledge remains compactly decodable despite representational reorganization. The results indicate that catastrophic forgetting is primarily an accessibility and manifold-alignment problem rather than information destruction.