Search papers, labs, and topics across Lattice.
This paper addresses the challenge of few-shot domain incremental learning (FSDIL) by introducing a novel algorithm called Continual Vision-Language Consolidation (CVLC), which effectively manages extreme data shortages. The method leverages latent space reservation in the base domain and employs dual coalescent projection (DCP) for parameter-efficient fine-tuning, allowing for seamless adaptation to new domains. Experimental results show that CVLC outperforms existing methods by up to 16% across various benchmark tasks, highlighting its effectiveness in overcoming data scarcity in DIL scenarios.
CVLC achieves a remarkable 16% performance boost in few-shot domain incremental learning, challenging the notion that more data is always necessary for effective adaptation.
Existing domain-incremental learning (DIL) strategies call for massive amounts of data to adapt to new domains and suffer from the overfitting problem in the case of data scarcity. This paper puts forward a relatively uncharted problem, namely, few-shot domain incremental learning (FSDIL), taking into account the problem of extreme data shortages in the realm of DIL. A novel algorithm, namely Continual Vision-Language Consolidation (CVLC), is proposed to address the FSDIL problem, where the key idea lies in the concept of latent space reservation in the base domain coupled with dual coalescent projection (DCP) as a parameter-efficient fine-tuning method. First, the vision prototype is calibrated while multiple templates and synonyms are generated via LLMs to induce the language prototype. The vision and language prototypes are fused. Adaptation to never-ending arrivals of new domains is done by the DCP technique, fine-tuned in such a way to prepare the model to unseen domains via latent-space reservations committed in the base domain. CVLC is structured under shared and domain-specific components to combine general knowledge and domain-specific details. The advantage of our approach is demonstrated through a range of benchmark problems and comparisons with prior arts, in which CVLC outperforms them by up to a 16% gap. Our codes are shared publicly in https://github.com/Naeem-Paeedeh/CVLC .
