Training-Free Intelligibility-Guided Observation Addition for Noisy ASR

2026-02-24

AI Summary

This work addresses the significant performance degradation of automatic speech recognition (ASR) in noisy environments, where conventional speech enhancement frontends often introduce artifacts that impair recognition accuracy. To mitigate this issue without retraining either the enhancement or ASR models, the authors propose an untrained intelligibility-guided fusion strategy that dynamically combines noisy and enhanced speech signals. The method leverages intelligibility estimates derived directly from the ASR backend to generate frame-level or utterance-level fusion weights, enabling a lightweight and model-agnostic observation fusion mechanism. Experimental results demonstrate that the proposed approach consistently outperforms existing observation fusion baselines across diverse enhancement–ASR system combinations and datasets, exhibiting strong robustness and generalization capability.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Ensemble Methods
• Natural Language Processing: Speech

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Automatic speech recognition (ASR) degrades severely in noisy environments. Although speech enhancement (SE) front-ends effectively suppress background noise, they often introduce artifacts that harm recognition. Observation addition (OA) addressed this issue by fusing noisy and SE enhanced speech, improving recognition without modifying the parameters of the SE or ASR models. This paper proposes an intelligibility-guided OA method, where fusion weights are derived from intelligibility estimates obtained directly from the backend ASR. Unlike prior OA methods based on trained neural predictors, the proposed method is training-free, reducing complexity and enhances generalization. Extensive experiments across diverse SE-ASR combinations and datasets demonstrate strong robustness and improvements over existing OA baselines. Additional analyses of intelligibility-guided switching-based alternatives and frame versus utterance-level OA further validate the proposed design.

Problem

• noisy ASR
• speech enhancement
• observation addition
• intelligibility
• training-free

Innovation

• training-free
• intelligibility-guided
• observation addition
• noisy ASR
• speech enhancement