Search papers, labs, and topics across Lattice.
This paper introduces Semantic Text-Anchored Incremental Learning (STAIL), a novel framework designed to address catastrophic forgetting in deep learning models applied to medical imaging. By utilizing an asymmetric semantic consolidation buffer (SCB) that combines minimal image anchors with extensive textual descriptions, STAIL significantly reduces storage overhead while effectively reconstructing old tasks. Experimental results across diverse medical datasets demonstrate that STAIL enhances baseline performance, achieving an average improvement of 2.24% in sustained performance and 3.55% in reduced forgetting.
STAIL achieves significant performance gains in medical imaging tasks while drastically minimizing storage requirements and privacy concerns associated with traditional rehearsal methods.
Deep learning models applied to medical image analysis suffer from severe catastrophic forgetting when continually adapting to new clinical tasks in dynamic environments. Mainstream incremental learning methods typically mitigate this by rehearsing raw historical images. However, this pixel-level rehearsal incurs significant storage overhead, raises privacy concerns, and fails to adequately capture the true data distribution with sparse exemplars. Inspired by human cognitive mechanisms, we propose a novel framework termed Semantic Text-Anchored Incremental Learning (STAIL) for sequential clinical tasks. To overcome the rehearsal bottleneck, STAIL introduces an asymmetric semantic consolidation buffer (SCB). By incorporating a minimal set of image anchors and extensive textual descriptions, the SCB enables dense semantic reconstruction of old tasks at a minimal storage cost. Furthermore, we design an LLM-derived Semantic Anchoring Mechanism (LSAM) that leverages the stable semantic space of frozen large language models as developmental priors. This mechanism explicitly anchors evolving visual features to textual representations, guiding and constraining plasticity and stability at both macroscopic and microscopic levels. Extensive experiments across three heterogeneous medical datasets, covering fundus, ultrasound, and X-ray imaging, demonstrate that STAIL acts as a highly effective plug-and-play module. It comprehensively enhances the performance of various existing baselines, achieving average gains of 2.24\% in AAA-AUC for sustained performance and 3.55\% in BWT-AUC for reduced forgetting. Code is available.
