Search papers, labs, and topics across Lattice.
This paper addresses the prompt collapse problem in prompt-based continual learning by introducing a novel framework that models prompts as probabilistic distributions. By constructing a mixture of these distributions and sampling from them, the proposed method captures diverse image patterns across tasks, enhancing the model's ability to learn from disjoint class subsets. Extensive experiments on standard benchmarks like ImageNet-R and CIFAR-100 validate the framework's effectiveness in preserving learned knowledge while adapting to new tasks.
Prompt-based continual learning can now capture diverse image distributions, overcoming the limitations of prompt collapse that hinder performance across tasks.
Continual learning aims to progressively learn from a sequence of tasks, each containing a disjoint subset of classes, while preserving previously learned knowledge. Prompt-based continual learning methods propose to learn a small set of parameters, i.e., prompts, by associating them with a query feature of an input image. These methods optimize the prompts, attempting to represent diverse patterns of images. However, we have observed that existing prompt-based methods suffer from a prompt collapse problem, that is, the prompts tend to be highly similar to each other, thereby failing to capture the diverse data distributions in continual learning scenarios. To address this issue, we propose in this paper a novel prompt-based continual learning framework that captures diverse patterns of images across a sequence of tasks. To this end, we model each prompt as a probabilistic distribution and construct a mixture of these distributions, from which we sample diverse prompts. This enables our model to effectively capture highly diverse image distributions in the continual learning process. We also present a distribution regularization loss to prevent abrupt changes in the prompt distributions throughout the training process. We show extensive experimental results for continual learning on standard benchmarks, including ImageNet-R, CIFAR-100, and CUB-200, demonstrating the effectiveness of our framework.
