NeuralMUSIC: A Hybrid Neural-Subspace Framework for Robot Sound Source Localization

2026-06-17

AI Summary

This work proposes a neural-subspace hybrid framework for sound source localization to address the performance degradation of conventional MUSIC algorithms under low signal-to-noise ratios and the limited generalization capability of purely data-driven approaches. The method leverages a neural network to estimate the spatial covariance matrix from multi-channel microphone signals and integrates this estimate into the MUSIC pipeline. A frequency-domain attention mechanism is employed to fuse information across multiple frequency bands, while a self-supervised spatial correlation learning strategy enhances the utilization of unlabeled data and improves cross-domain generalization. Experimental results demonstrate that the proposed approach significantly outperforms existing techniques across diverse robotic auditory scenarios, achieving high accuracy, strong robustness, and excellent adaptability to unseen environments.

Abstract

Reliable sound source localization is fundamental to robot audition, enabling autonomous robots to perceive spatial cues and operate effectively in dynamic environments. Classical methods such as Multiple Signal Classification (MUSIC) offer strong theoretical foundations but degrade under low signal-to-noise ratios. While deep learning-based approaches achieve promising performance, they often struggle with limited generalization across conditions. To address these challenges, we propose NeuralMUSIC, a hybrid neural-subspace framework for robotic sound source localization. Specifically, a neural network first estimates the spatial covariance matrix from multichannel microphone observations. The predicted covariance is then integrated into a classical MUSIC pipeline with eigenvalue decomposition (EVD) and pseudo-spectrum computation, followed by a Frequency Attention Fusion (FAF) module to produce the final DOA estimates. To improve data efficiency, we further introduce a Self-supervised Spatial Correlation Learning (SSCL) strategy that leverages unlabeled acoustic data to capture spatial structure. Extensive experiments across different robotic tasks demonstrate that NeuralMUSIC achieves competitive localization accuracy while exhibiting improved robustness and cross-domain generalization.

Problem

• sound source localization
• robot audition
• generalization
• low signal-to-noise ratio
• spatial cues

Innovation

• NeuralMUSIC
• sound source localization
• hybrid neural-subspace
• self-supervised learning
• MUSIC algorithm