Search papers, labs, and topics across Lattice.
This paper introduces CLeaRS, a new benchmark for continual vision-language learning in remote sensing, comprising 10 subsets with over 207k image-text pairs and three evaluation protocols. Benchmarking various vision-language models on CLeaRS reveals significant catastrophic forgetting across long-horizon, modality-incremental, and task-incremental settings. The study also demonstrates that existing continual learning methods struggle to effectively mitigate forgetting when applied to remote sensing VLMs, highlighting the need for specialized techniques.
Remote sensing VLMs forget everything they've learned when exposed to new tasks or modalities, and existing continual learning methods don't help.
Current remote sensing vision-language models (RS VLMs) demonstrate impressive performance in image interpretation but rely on static training data, limiting their ability to accommodate continuously emerging sensing modalities and downstream tasks. This exposes a fundamental challenge: enabling RS VLMs to continually adapt without catastrophic forgetting. Despite its practical importance, the continual learning capability of RS VLMs remains underexplored, and no dedicated benchmark currently exists. In this work, we present CLeaRS, a comprehensive benchmark for continual vision-language learning in remote sensing. CLeaRS comprises 10 curated subsets with over 207k image-text pairs, spanning diverse interpretation tasks, sensing modalities, and application scenarios. We further define three evaluation protocols: long-horizon, modality-incremental, and task-incremental settings, to systematically assess continual adaptation. Extensive benchmarking of diverse vision-language models reveals catastrophic forgetting across all settings. Moreover, representative continual learning methods, when adapted to RS VLMs, exhibit limited effectiveness in handling task, instruction, and modality transitions. Our findings underscore the need for developing continual learning methods tailored to RS VLMs.
