Search papers, labs, and topics across Lattice.
This paper introduces NeuralMUSIC, a hybrid neural-subspace framework that enhances robot sound source localization by integrating a neural network for spatial covariance estimation with classical MUSIC techniques. By addressing the limitations of traditional methods under low signal-to-noise ratios and improving generalization through a Self-supervised Spatial Correlation Learning strategy, NeuralMUSIC demonstrates significant advancements in localization accuracy and robustness across various robotic tasks. The results indicate that this approach not only performs competitively but also generalizes better across different acoustic conditions compared to existing methods.
NeuralMUSIC achieves superior sound source localization accuracy and robustness by seamlessly combining deep learning with classical techniques, even in challenging acoustic environments.
Reliable sound source localization is fundamental to robot audition, enabling autonomous robots to perceive spatial cues and operate effectively in dynamic environments. Classical methods such as Multiple Signal Classification (MUSIC) offer strong theoretical foundations but degrade under low signal-to-noise ratios. While deep learning-based approaches achieve promising performance, they often struggle with limited generalization across conditions. To address these challenges, we propose NeuralMUSIC, a hybrid neural-subspace framework for robotic sound source localization. Specifically, a neural network first estimates the spatial covariance matrix from multichannel microphone observations. The predicted covariance is then integrated into a classical MUSIC pipeline with eigenvalue decomposition (EVD) and pseudo-spectrum computation, followed by a Frequency Attention Fusion (FAF) module to produce the final DOA estimates. To improve data efficiency, we further introduce a Self-supervised Spatial Correlation Learning (SSCL) strategy that leverages unlabeled acoustic data to capture spatial structure. Extensive experiments across different robotic tasks demonstrate that NeuralMUSIC achieves competitive localization accuracy while exhibiting improved robustness and cross-domain generalization.
