Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper addresses the performance limitations of deep spiking neural networks (SNNs) that typically rely on batch normalization (BN), which incurs additional runtime costs. By identifying catastrophic firing-rate decay as a key issue in normalization-free SNNs, the authors introduce the Intrinsically Stable SNN (IS-SNN) architecture, which eliminates activation-normalization layers through topology-aware weight standardization and modified residual connections. Comprehensive experiments demonstrate that IS-SNN not only matches but often surpasses the performance of traditional dynamic BN methods, achieving a competitive accuracy of 68.05% on ImageNet while significantly reducing FPGA resource consumption by 96.4%.
IS-SNN achieves state-of-the-art accuracy on ImageNet without the computational overhead of batch normalization, revolutionizing the efficiency of spiking neural networks.
The performance of deep spiking neural networks (SNNs) often relies on batch normalization (BN). However, the advanced dynamic BN variants used in state-of-the-art models introduce runtime multiplications, which weaken the hardware-efficiency motivation of SNNs. To address this tension, we identify catastrophic firing-rate decay as a primary cause of severe performance degradation in normalization-free SNNs. Guided by this insight, this work proposes the Intrinsically Stable SNN (IS-SNN) architecture, which removes activation-normalization layers by enforcing signal homeostasis through topology-aware weight standardization and modified residual connections. By folding the standardization operations into static weights offline, IS-SNN removes the runtime statistics tracking and multiplications introduced by activation normalization, restoring an accumulation-oriented inference datapath. Comprehensive experiments show that IS-SNN achieves performance competitive with or superior to computationally expensive dynamic BN techniques across VGG, ResNet, and Transformer-based models. Notably, it achieves a competitive accuracy of 68.05\% on ImageNet and overcomes the severe depth limitations of prior BN-free attempts. Together with a 96.4\% reduction in FPGA lookup table resource consumption for neuron implementations, these results support IS-SNN as a practical framework for building accurate and hardware-friendly deep neuromorphic systems.
Tsinghua AI