Search papers, labs, and topics across Lattice.
This paper introduces a novel approach to generating Room Impulse Responses (RIRs) by fine-tuning a pre-trained text-to-audio model. To overcome the scarcity of paired text-RIR data, the authors create a labeling pipeline using vision-language models to extract acoustic descriptions from image-RIR datasets. Experiments demonstrate that the fine-tuned model generates plausible RIRs, validated through MUSHRA listening tests and downstream ASR performance improvements, showing the effectiveness for speech data augmentation.
Unlock realistic acoustic simulations with a text prompt: fine-tuning a text-to-audio model generates plausible room impulse responses, even with limited paired data.
Room Impulse Responses (RIRs) enable realistic acoustic simulation, with applications ranging from multimedia production to speech data augmentation. However, acquiring high-quality real-world RIRs is labor-intensive, and data scarcity remains a challenge for data-driven RIR generation approaches. In this paper, we propose a novel approach to RIR generation by fine-tuning a pre-trained text-to-audio model, demonstrating for the first time that large-scale generative audio priors can be effectively leveraged for the task. To address the lack of text-RIR paired data, we establish a labeling pipeline utilizing vision-language models to extract acoustic descriptions from existing image-RIR datasets. We introduce an in-context learning strategy to accommodate free-form user prompts during inference. Evaluations involving MUSHRA listening tests and downstream ASR performance demonstrate that our model generates plausible RIRs and serves as an effective tool for speech data augmentation.
