Single-Channel Target Speech Extraction Utilizing Distance and Room Clues

2025-05-20

AI Summary

Single-channel target speech extraction (TSE) suffers from poor generalization across acoustic environments—particularly due to overreliance on source distance cues, which fail to adapt to varying room dimensions and reverberation characteristics. To address this, we propose a joint modeling framework that simultaneously encodes distance and room information. Specifically, we introduce differentiable embeddings for both source distance and room acoustics (i.e., room dimensions and reverberation time), enabling distance–environment dual-driven TSE in the time-frequency domain. Furthermore, we incorporate DRR-aware feature enhancement to improve discriminability of acoustic cues. Extensive experiments on both simulated and real-recorded datasets demonstrate significant improvements in separation performance and cross-room robustness, validating the efficacy of joint distance–room modeling. The source code and an interactive demo are publicly available.

Technology Category

• Machine Learning: Multimodal Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

This paper aims to achieve single-channel target speech extraction (TSE) in enclosures utilizing distance clues and room information. Recent works have verified the feasibility of distance clues for the TSE task, which can imply the sound source's direct-to-reverberation ratio (DRR) and thus can be utilized for speech separation and TSE systems. However, such distance clue is significantly influenced by the room's acoustic characteristics, such as dimension and reverberation time, making it challenging for TSE systems that rely solely on distance clues to generalize across a variety of different rooms. To solve this, we suggest providing room environmental information (room dimensions and reverberation time) for distance-based TSE for better generalization capabilities. Especially, we propose a distance and environment-based TSE model in the time-frequency (TF) domain with learnable distance and room embedding. Results on both simulated and real collected datasets demonstrate its feasibility. Demonstration materials are available at https://runwushi.github.io/distance-room-demo-page/.

Problem

• Achieve single-channel target speech extraction using distance clues
• Improve generalization across rooms by incorporating room information
• Propose learnable distance and room embedding for better performance

Innovation

• Utilizes distance clues for speech extraction
• Incorporates room environmental information
• Proposes learnable distance and room embedding