Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates the application of spiking and event-driven neuromorphic models to enhance the efficiency of automatic speech recognition (ASR) systems, specifically focusing on the SpeechMamba model. By implementing an event-driven SpeechMamba with FATReLU activation, the authors achieve over 60% activation sparsity with minimal accuracy loss, while a spiking variant attains over 70% sparsity and reduces parameter count by 30% compared to existing SNNs. Additionally, the development of a cycle-accurate event-driven simulator facilitates algorithm-hardware co-exploration, leading to over 10% improvements in computational efficiency.
Achieving over 70% activation sparsity in speech recognition models could revolutionize real-time interaction on edge devices.
Deep learning has greatly advanced automatic speech recognition (ASR), enabling widespread deployment on edge devices such as smartphones and smart home systems. However, the computational and energy demands of deep neural networks pose significant challenges for such resource-constrained deployments, introducing latency and limiting real-time interaction. Neuromorphic computing offers a promising solution by introducing activation sparsity through spiking neural networks (SNNs) and event-driven neural networks, converting dense operations into sparse computations. However, a study that evaluates the hardware benefits of different neuromorphic strategies remains lacking for ASR. This paper explores spiking and event-driven neuromorphic neural networks to improve activation sparsity in the state-of-the-art SpeechMamba model for ASR. We introduce an event-driven SpeechMamba with FATReLU activation, achieving over 60% activation sparsity with less than 1% accuracy degradation on LibriSpeech. We also propose a spiking SpeechMamba that attains over 70% sparsity while using 30% fewer parameters than comparable SNNs. Finally, we develop a cycle-accurate event-driven simulator enabling flexible algorithm-hardware co-exploration, which helps us identify computational bottlenecks and yields over 10% additional efficiency improvements.