Search papers, labs, and topics across Lattice.
CATFormer, a novel spiking neural network (SNN) architecture, is introduced to address catastrophic forgetting in class-incremental learning (CIL). The architecture employs a Dynamic Threshold Leaky Integrate-and-Fire (DTLIF) neuron model with context-adaptive thresholds for knowledge retention, coupled with a Gated Dynamic Head Selection (G-DHS) mechanism for task-agnostic inference. Experiments on static and neuromorphic datasets demonstrate that CATFormer surpasses existing rehearsal-free CIL algorithms, making it suitable for energy-efficient incremental learning.
By dynamically modulating neuronal excitability in spiking neural networks, CATFormer achieves state-of-the-art performance in class-incremental learning, outperforming existing methods on both static and neuromorphic datasets.
Although deep neural networks perform extremely well in controlled environments, they fail in real-world scenarios where data isn't available all at once, and the model must adapt to a new data distribution that may or may not follow the initial distribution. Previously acquired knowledge is lost during subsequent updates based on new data. a phenomenon commonly known as catastrophic forgetting. In contrast, the brain can learn without such catastrophic forgetting, irrespective of the number of tasks it encounters. Existing spiking neural networks (SNNs) for class-incremental learning (CIL) suffer a sharp performance drop as tasks accumulate. We here introduce CATFormer (Context Adaptive Threshold Transformer), a scalable framework that overcomes this limitation. We observe that the key to preventing forgetting in SNNs lies not only in synaptic plasticity but also in modulating neuronal excitability. At the core of CATFormer is the Dynamic Threshold Leaky Integrate-and-Fire (DTLIF) neuron model, which leverages context-adaptive thresholds as the primary mechanism for knowledge retention. This is paired with a Gated Dynamic Head Selection (G-DHS) mechanism for task-agnostic inference. Extensive evaluation on both static (CIFAR-10/100/Tiny-ImageNet) and neuromorphic (CIFAR10-DVS/SHD) datasets reveals that CATFormer outperforms existing rehearsal-free CIL algorithms across various task splits, establishing it as an ideal architecture for energy-efficient, true-class incremental learning.
