Search papers, labs, and topics across Lattice.
This paper introduces SelectTSL, an end-to-end architecture for prompt-guided selective target sound localization that addresses the limitations of existing deep learning methods which localize all active sources without selectivity. By employing a Prompt-Guided Selective Attention Module (PGSA) to create prompt-informed embeddings, SelectTSL enhances inter-channel phase differences to accurately estimate the direction of arrival (DoA) and the cardinality of target sound sources in complex acoustic scenes. Experimental results show that SelectTSL significantly outperforms baseline methods and demonstrates strong generalization capabilities in real-world environments.
SelectTSL achieves precise localization of user-specified target sounds in complex acoustic environments, outperforming traditional methods that lack selectivity.
Humans can selectively attend to a target sound and estimate its direction in complex scenarios, whereas such selective localization remains challenging for current deep learning-based systems. Sound source localization (SSL) has achieved remarkable success with deep learning, yet most methods localize all active sources without selectivity. Conversely, target sound extraction (TSE) extracts sources using multimodal prompts but typically fails to preserve the multichannel spatial information required for accurate localization. To bridge this gap, we formulate the task of prompt-guided selective target sound localization and propose SelectTSL, an end-to-end architecture that localizes only the user-specified target in multi-source acoustic scenes. Specifically, we design a target-aware selective localization strategy that employs a Prompt-Guided Selective Attention Module (PGSA) to generate prompt-informed embeddings. These embeddings guide an inter-channel phase difference (IPD) enhancer to refine raw phase cues, fusing with target magnitudes to jointly estimate direction of arrival (DoA) and target-source cardinality, i.e., the number of target sound sources. This coupled design effectively focuses on the user-specified target spatial cues for selective localization and also handles time-varying numbers of target sources. Extensive experiments on both synthetic data and real-world recordings demonstrate that our proposed method consistently outperforms other baselines and exhibits robust generalization to real acoustic environments.
