Search papers, labs, and topics across Lattice.
This paper introduces Black-CL, a benchmark for Vision-Language Models (VLMs) that addresses the challenges of continual learning in cloud-hosted environments by enforcing constraints on weight and architecture accessibility, computation limits, and task-agnostic inference. The authors propose BETA, a novel approach that optimizes textual prototypes through three innovative components: Semantic Projection Accumulation, Latent Distribution Replay, and Test-Time Prototype Adaptation, enabling effective knowledge retention without traditional backpropagation methods. Experimental results across ten datasets reveal that BETA not only outperforms existing black-box tuners but also achieves competitive performance with a fraction of the parameters compared to white-box continual learning methods.
BETA achieves state-of-the-art continual learning performance with just 0.05 million trainable parameters, outperforming traditional methods by 180–3000 times in parameter efficiency.
The rapid deployment of Vision-Language Models (VLMs) in dynamic environments necessitates the ability to learn continuously without forgetting. However, traditional continual learning (CL) settings often rely on white-box paradigms, which is increasingly invalidated by the shift toward cloud-hosted models. In this paper, we introduce Black-CL, a more realistic benchmark for VLMs that enforces three primary real-world challenges: weight and architecture inaccessibility, constrained computation, and task-agnostic inference. The learner can query only output embeddings or logits, with no gradient flow through or structural modification of the backbone. Current CL methodologies, which rely on backbone backpropagation or complex parameter expansion, are fundamentally incompatible with these constraints. Under this setting, we propose BETA, a simple yet effective baseline built on the key insight that solely optimizing textual prototypes can navigate the complexities of CL. BETA integrates three core components: Semantic Projection Accumulation (SPA) for incremental knowledge acquisition, Latent Distribution Replay (LDR) for anchoring the embedding space against catastrophic forgetting, and Test-Time Prototype Adaptation (TTPA) for dynamic, instance-aware boundary refinement. Extensive experiments across ten diverse datasets and various backbones demonstrate that BETA significantly outperforms existing black-box tuners. Remarkably, with only 0.05 M trainable parameters, a 180--3000$\times$ reduction compared to competitive methods, BETA achieves performance on par with or even exceeding white-box CL methods. We believe Black-CL and BETA provide a foundational framework for future advancements in continual learning and accelerates the transition of continual learning from academia to real-world systems.
