Single-Channel Distance-Based Source Separation for Mobile GPU in Outdoor and Indoor Environments

2025-01-06

AI Summary

This work addresses the challenges of distance-unaware separation and poor deployability on mobile devices for single-channel speech separation in complex real-world acoustic environments—both indoor and outdoor. Methodologically, it introduces the first distance-aware source separation framework tailored for realistic scenarios: (i) extending distance-based separation to non-ideal outdoor conditions; (ii) proposing a Relation-aware Self-Attention (RSA) mechanism that models linear geometric relationships between distance and spectral features, preserving physical interpretability while enhancing contextual modeling efficiency; and (iii) adopting a two-stage Conformer architecture accelerated via TensorFlow Lite GPU delegation for on-device inference. Experiments demonstrate state-of-the-art separation performance across both indoor and outdoor settings, alongside significant improvements in mobile energy efficiency and real-time inference speed. The proposed approach establishes a new paradigm for robust, edge-deployable speech separation.

Technology Category

• Machine Learning: Hardware-aware ML
• Natural Language Processing: Speech
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

This study emphasizes the significance of exploring distance-based source separation (DSS) in outdoor environments. Unlike existing studies that primarily focus on indoor settings, the proposed model is designed to capture the unique characteristics of outdoor audio sources. It incorporates advanced techniques, including a two-stage conformer block, a linear relation-aware self-attention (RSA), and a TensorFlow Lite GPU delegate. While the linear RSA may not capture physical cues as explicitly as the quadratic RSA, the linear RSA enhances the model's context awareness, leading to improved performance on the DSS that requires an understanding of physical cues in outdoor and indoor environments. The experimental results demonstrated that the proposed model overcomes the limitations of existing approaches and considerably enhances energy efficiency and real-time inference speed on mobile devices.

Problem

• Audio Source Separation
• Mobile Graphics Processor
• Indoor and Outdoor Environment

Innovation

• Mobile GPU
• Monoaural Sound Source Separation
• Outdoor Environment Optimization