Search papers, labs, and topics across Lattice.
This study addresses the challenge of limited training data in singing voice separation by leveraging domain adaptation from established speech enhancement models. Two fine-tuning strategies鈥攆ull fine-tuning and Low-Rank Adaptation (LoRA)鈥攚ere evaluated, revealing that while full fine-tuning enhances singing voice separation performance, it risks catastrophic forgetting of speech enhancement capabilities. In contrast, LoRA fine-tuning maintains competitive performance with minimal parameter increase, showcasing its effectiveness in data-scarce environments.
Adapting speech enhancement models can significantly boost singing voice separation performance without sacrificing their original capabilities.
State-of-the-art speech enhancement models benefit from large-scale labeled datasets, whereas singing voice separation models suffer from limited available training data. To address this limitation, we formulate singing voice separation as domain adaptation from speech enhancement to singing voice separation. We investigate two fine-tuning strategies: full fine-tuning and parameter-efficient fine-tuning using Low-Rank Adaptation (LoRA) on a discriminative and a generative model. Models with either adaptation strategy outperform the same architectures trained from scratch by 0.29-1.8 dB in Signal-to-Distortion-Ratio. Full fine-tuning yields the highest singing voice separation performance, but catastrophic forgetting degrades speech enhancement performance. LoRA fine-tuning achieves competitive singing voice separation performance while preserving the original speech enhancement capability with only 6-12% additional parameters compared to the base speech enhancement model. Furthermore, the generative model shows improved generalization to an unseen test set. The results demonstrate that adapting pretrained speech enhancement models is an effective strategy for training singing voice separation models in data-scarce scenarios.
