Search papers, labs, and topics across Lattice.
This paper introduces Miles, a novel approach for Class Incremental Learning (CIL) that leverages pre-trained models to efficiently expand the parameter space without incurring high computational costs or catastrophic forgetting. By decoupling learnable modules from the pre-trained backbone and utilizing intermediate feature information, Miles enables a guided optimization process that clusters new categories effectively while maintaining metric equilibrium. Experimental results across six benchmark datasets show that Miles outperforms existing CIL methods, achieving state-of-the-art performance with a balanced trade-off between efficiency and effectiveness.
Miles achieves state-of-the-art performance in Class Incremental Learning by efficiently expanding the parameter space without sacrificing computational efficiency or risking catastrophic forgetting.
Class Incremental Learning (CIL) aims to learn new concepts consistently from a data stream without forgetting. Unlike typical CIL methods which need to learn a model from scratch, pre-trained model (PTM) can easily adapt to a new task with fine-tuning. However, existing PTM-based CIL methods fail to achieve a trade-off between performance and computational expenditure, i.e., they either adopt the same parameter space so that leading catastrophic forgetting, or expand a new branch for each task but adding more computational cost. To this end, we propose MetrIc Learning with Expandable Subspace (Miles) to harness the prior information within pre-trained knowledge, thereby orchestrating an efficient expansion of the parameter space through guided optimization. Specifically, it decouples the learnable modules with the pre-trained model, exploiting prior information from intermediate features of the backbone network to enable more flexible parameter expansion. Then, a central loss is adopted to guide the new category to cluster towards the corresponding prototype in the new task subspace while incorporating an auxiliary distance regularization term to maintain metric equilibrium across tasks. Extensive experiments on six benchmark datasets demonstrate that Miles achieves state-of-the-art performance in various CIL settings.
