SelectTSL: Prompt-Guided Selective Target Sound Localization in Complex Scenarios

2026-07-02

AI Summary

Existing deep learning approaches struggle to accurately localize target sounds in complex, multi-source acoustic environments based on user-provided prompts, primarily due to their limited ability to focus on goal-oriented spatial cues. This work proposes SelectTSL, an end-to-end architecture that achieves prompt-driven selective sound source localization for the first time. It introduces a Prompt-Guided Selective Attention module to generate target-aware embeddings, which guide the enhancement of inter-channel phase differences and the fusion of multi-channel beamforming features for joint estimation of both the direction and the number of target sound sources. The method effectively handles dynamically varying numbers of sources while preserving full spatial information, significantly outperforming existing baselines on both synthetic and real-world datasets, and demonstrating superior generalization capability and localization accuracy.

Abstract

Humans can selectively attend to a target sound and estimate its direction in complex scenarios, whereas such selective localization remains challenging for current deep learning-based systems. Sound source localization (SSL) has achieved remarkable success with deep learning, yet most methods localize all active sources without selectivity. Conversely, target sound extraction (TSE) extracts sources using multimodal prompts but typically fails to preserve the multichannel spatial information required for accurate localization. To bridge this gap, we formulate the task of prompt-guided selective target sound localization and propose SelectTSL, an end-to-end architecture that localizes only the user-specified target in multi-source acoustic scenes. Specifically, we design a target-aware selective localization strategy that employs a Prompt-Guided Selective Attention Module (PGSA) to generate prompt-informed embeddings. These embeddings guide an inter-channel phase difference (IPD) enhancer to refine raw phase cues, fusing with target magnitudes to jointly estimate direction of arrival (DoA) and target-source cardinality, i.e., the number of target sound sources. This coupled design effectively focuses on the user-specified target spatial cues for selective localization and also handles time-varying numbers of target sources. Extensive experiments on both synthetic data and real-world recordings demonstrate that our proposed method consistently outperforms other baselines and exhibits robust generalization to real acoustic environments.

Problem

• sound source localization
• target sound extraction
• selective attention
• direction of arrival
• multimodal prompt

Innovation

• prompt-guided localization
• selective sound source localization
• inter-channel phase difference (IPD) enhancement
• direction of arrival (DoA) estimation
• target-source cardinality