Search papers, labs, and topics across Lattice.
This paper addresses the limitations of existing Music Source Separation (MSS) models that struggle with live recordings by introducing two novel datasets: CrowdioSet and PaRIRset. CrowdioSet enhances audio denoising through a combination of real ambience tracks and synthetic vocals, leading to improved separation performance in both objective and subjective evaluations. Additionally, PaRIRset, which consists of stereo impulse responses from 40 concert venues, significantly boosts MSS model performance beyond what is achievable with traditional real impulse responses from Speech Enhancement tasks.
Live music recordings can now be effectively separated using new datasets that account for venue acoustics and audience noise, transforming MSS capabilities.
Most Music Source Separation (MSS) models do not generalize well to live music recordings because they are trained on studio recordings alone, disregarding the venue acoustics, the speaker system's response and audience noise. We propose to bridge this gap by providing and training a model on two novel datasets. First, we present CrowdioSet: a noise dataset comprising 4800 real ambience tracks from Freesound and synthetic sing-alongs for the vocals in MUSDB18 and MOISESDB datasets, generated from zero-shot singing voice conversions. CrowdioSet enables effective audio denoising for live recordings, resulting in superior separation both in objective and subjective evaluations. Second, we introduce PaRIRset, a stereo impulse response dataset captured across 40 professional concert venues using a microphone array. Our results show that adding PaRIRset RIRs increases the performance of a MSS model compared to using real RIRs from Speech Enhancement tasks alone. We make the examples, code, model weights, PaRIRset, and CrowdioSet freely available to the public.
