Track synthesis, materials and catalysis as it is published.
Search papers, labs, and topics across Lattice.
This paper introduces SELECT, a novel method for Class-Incremental Semantic Segmentation (CISS) that addresses the challenges of catastrophic forgetting and background shift by leveraging selective context transfer. By grounding new classes in semantically similar past classes through a Context Transfer Attention mechanism, SELECT enhances the initialization of new class representations while mitigating the risk of information dilution. Experimental results demonstrate that SELECT achieves significant improvements in mean Intersection over Union (mIoU), outperforming existing methods by 2.2% on Pascal VOC and 2.8% on ADE20K, effectively balancing stability and plasticity in CISS.
SELECT achieves a remarkable 2.8% boost in mIoU for class-incremental segmentation by intelligently transferring context from semantically related classes.
Class-Incremental Semantic Segmentation (CISS) is fundamentally challenged by catastrophic forgetting and background shift, where learning new concepts degrades performance on previously seen classes. While existing methods attempt to balance stability (retaining old knowledge) and plasticity (learning new knowledge), they often fail to leverage prior knowledge effectively. These approaches typically rely on indiscriminate knowledge transfer or ambiguous initializations, which can dilute crucial semantic information. To overcome this limitation, we propose SELECT, a novel approach for Selective Context Transfer, which instead grounds each new class in a small set of semantically similar past classes. Its core is a Context Transfer Attention mechanism that aggregates the learned tokens from similar classes into a structured initialization for the new class. To ensure this transfer does not corrupt the borrowed representations, we add a controlled noise perturbation and a margin-based context-transfer loss that enforces separation between the new class token and its source tokens. Extensive experiments on Pascal VOC and ADE20K show that SELECT consistently outperforms prior work, achieving mIoU of 2.2% on VOC and 2.8% on ADE, providing an effective handle on the stability-plasticity dilemma. Code is available at https://github.com/avigupta2798/SELECT.
Microsoft Research