Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a novel parameter-efficient continual learning (PECL) method tailored for automatic speech recognition (ASR) that mitigates catastrophic forgetting during sequential fine-tuning. By partitioning pretrained weight matrices into head and tail subspaces and restricting adaptation to the low-energy tail subspace, the approach preserves essential model components while enabling effective task adaptation. Experimental results on two benchmarks reveal that this method significantly reduces forgetting and outperforms existing PECL baselines, underscoring its potential for robust ASR applications.
Forgetting in ASR models can be drastically reduced by focusing adaptation on low-energy subspaces, leading to superior performance across tasks.
Speech foundation models enable strong general-purpose ASR and are attractive for downstream adaptation. However, their size and the catastrophic forgetting induced by sequential fine-tuning demand parameter-efficient and regularized training methods, motivating parameter-efficient continual learning (PECL). While PECL has been widely studied in NLP and vision, it has received less attention in ASR. In this paper, we propose a simple yet effective PECL method based on recent advances in parameter-efficient fine-tuning for ASR. We partition pretrained weight matrices into head and tail subspaces according to singular values and restrict adaptation to approximate rotations within the low-energy tail subspace, preserving dominant components and reducing forgetting. For subsequent tasks, rotations are combined via weight averaging to further improve retention. Experiments on two benchmarks demonstrate reduced forgetting and superior overall performance compared to recent PECL baselines.