Search papers, labs, and topics across Lattice.
This paper introduces sLoTh, a parameter-efficient continual learning framework designed for pretrained sparse event-based vision transformers, which allows for adaptation in dynamic environments while conserving memory and energy. By freezing the backbone and implementing low-rank attention updates alongside shared neuronal threshold modulation, sLoTh updates less than 1% of model parameters, eliminating the need for replay buffers. Experimental results show that sLoTh achieves competitive performance in class-incremental and online continual learning tasks across multiple datasets, with a significant reduction in energy consumption compared to traditional dense architectures.
sLoTh enables continual learning in sparse event-based transformers with less than 1% parameter updates, achieving competitive performance while slashing energy consumption by 6.5x.
Robotic and edge intelligence systems operate in dynamic environments where data arrives continuously, requiring models to adapt while preserving previously learned knowledge under strict memory and energy constraints. While parameter-efficient fine-tuning has shown promise for continual learning with vision transformers, conventional architectures rely on dense computation and remain costly for real-world deployment. Sparse event-based vision transformers provide energy-efficient event-driven computation, yet their continual learning capabilities remain largely unexplored. We here introduce sLoTh, a parameter-efficient continual learning framework for pretrained sparse event-based (spiking) vision transformers. sLoTh freezes the backbone and restricts plasticity to scalable-efficient low-rank attention updates (seLoRA) and shared neuronal threshold modulation, enabling adaptation without replay buffers by updating less than 1% of model parameters. Experiments across CIFAR-100, Tiny-ImageNet, ImageNet-100, and ImageNet-R with up to 100 tasks demonstrate competitive rehearsal-free performance in class-incremental learning and online continual learning, while enabling approximately 6.5x lower energy consumption than conventional dense vision transformers.
