Search papers, labs, and topics across Lattice.
This paper introduces MambAdapter, a novel parameter-efficient transfer learning method that integrates the Mamba state-space model into low-rank bottleneck adapters for audio and speech processing. By leveraging parameter sharing and a lightweight Mamba module, MambAdapter effectively models audio features while significantly reducing computational and memory costs. The results show that MambAdapter matches or surpasses existing PETL baselines across multiple audio classification tasks and speech recognition languages, even with fewer parameters.
MambAdapter achieves superior performance in audio and speech tasks while drastically cutting down on computational resources.
Fine-tuning Transformer-based foundation models has become the dominant strategy for domain adaptation in audio and speech processing. To reduce the computational and memory costs of this process, parameter-efficient transfer learning (PETL) methods have been widely explored. Meanwhile, Mamba, a recent state-space model, has emerged as a promising alternative to Transformers for sequence modeling. In this work, we present MambAdapter, a parameter-efficient transfer learning approach that integrates Mamba into low-rank bottleneck adapters. Our design combines parameter sharing across adapters with the injection of a lightweight Mamba module, enabling more effective modeling of audio features. We demonstrate that MambAdapter matches or outperforms strong PETL baselines on four audio classification tasks and five speech recognition languages, even when operating under reduced parameter budgets.
Mila