Search papers, labs, and topics across Lattice.
This paper introduces Quantum-Gated Task-interaction Knowledge Distillation (QKD), a novel framework for class-incremental learning that uses quantum gating to model relational dependencies among task embeddings and guide inter-task knowledge transfer. QKD dynamically captures sample-to-task relevance using a quantum-gated task modulation mechanism, facilitating knowledge distillation from old to new adapters based on task-embedding-level correlation weights. Experiments demonstrate that QKD effectively mitigates catastrophic forgetting and achieves state-of-the-art performance in class-incremental learning.
Quantum-inspired gating unlocks better knowledge transfer in class-incremental learning, outperforming existing methods by dynamically modeling task relationships.
Class-incremental learning (CIL) aims to continuously accumulate knowledge from a stream of tasks and construct a unified classifier over all seen classes. Although pretrained models (PTMs) have shown promising performance in CIL, they still struggle with the entanglement of multi-task subspaces, leading to catastrophic forgetting when task routing parameters are poorly calibrated or task-level representations are rigidly fixed. To address this issue, we propose a novel Quantum-Gated Task-interaction Knowledge Distillation (QKD) framework that leverages quantum gating to guide inter-task knowledge transfer. Specifically, we introduce a quantum-gated task modulation gating mechanism to model the relational dependencies among task embedding, dynamically capturing the sample-to-task relevance for both joint training and inference across streaming tasks. Guided by the quantum gating outputs, we perform task-interaction knowledge distillation guided by these task-embedding-level correlation weights from old to new adapters, enabling the model to bridge the representation gaps between independent task subspaces. Extensive experiments demonstrate that QKD effectively mitigates forgetting and achieves state-of-the-art performance.
