Search papers, labs, and topics across Lattice.
The paper introduces Image2AVScene, a novel task of generating 3D audio-visual scenes from single images, and proposes SonoWorld, a framework to address this task. SonoWorld generates a 360 panorama, lifts it into a 3D scene, places language-guided sound anchors, and renders ambisonics, aligning spatial audio with scene geometry and semantics. Experiments on a new real-world dataset and user studies demonstrate the effectiveness of SonoWorld, with applications in audio-visual rendering, one-shot acoustic learning, and spatial source separation.
Now you can turn a single image into a navigable 3D world complete with spatial audio, opening the door to richer immersive experiences.
Tremendous progress in visual scene generation now turns a single image into an explorable 3D world, yet immersion remains incomplete without sound. We introduce Image2AVScene, the task of generating a 3D audio-visual scene from a single image, and present SonoWorld, the first framework to tackle this challenge. From one image, our pipeline outpaints a 360{\deg} panorama, lifts it into a navigable 3D scene, places language-guided sound anchors, and renders ambisonics for point, areal, and ambient sources, yielding spatial audio aligned with scene geometry and semantics. Quantitative evaluations on a newly curated real-world dataset and a controlled user study confirm the effectiveness of our approach. Beyond free-viewpoint audio-visual rendering, we also demonstrate applications to one-shot acoustic learning and audio-visual spatial source separation. Project website: https://humathe.github.io/sonoworld/
