Search papers, labs, and topics across Lattice.
The paper introduces Woosh, a sound effect foundation model comprising a high-quality audio encoder/decoder, a text-audio alignment model, and text-to-audio/video-to-audio generative models. Woosh is optimized for sound effects generation, addressing a need for open generative models in the audio research community. Evaluations against StableAudio-Open and TangoFlux demonstrate competitive or superior performance across modules, with distilled versions enabling low-resource operation.
Woosh leapfrogs existing open models in sound effect generation, offering researchers a new high-quality foundation model and paving the way for more realistic and immersive audio experiences.
The audio research community depends on open generative models as foundational tools for building novel approaches and establishing baselines. In this report, we present Woosh, Sony AI's publicly released sound effect foundation model, detailing its architecture, training process, and an evaluation against other popular open models. Being optimized for sound effects, we provide (1) a high-quality audio encoder/decoder model and (2) a text-audio alignment model for conditioning, together with (3) text-to-audio and (4) video-to-audio generative models. Distilled text-to-audio and video-to-audio models are also included in the release, allowing for low-resource operation and fast inference. Our evaluation on both public and private data shows competitive or better performance for each module when compared to existing open alternatives like StableAudio-Open and TangoFlux. Inference code and model weights are available at https://github.com/SonyResearch/Woosh. Demo samples can be found at https://sonyresearch.github.io/Woosh/.
