Phonetically-Augmented Discriminative Rescoring for Voice Search Error Correction

2025-06-06

AI Summary

End-to-end (E2E) automatic speech recognition (ASR) suffers from low recognition accuracy for rare movie titles in voice search, primarily due to insufficient coverage of long-tail vocabulary in training data and high annotation costs. To address this, we propose a phoneme-augmented discriminative rescoring method: first generating a phoneme-similar candidate set from ASR outputs, then jointly modeling acoustic confidence, phoneme sequence matching scores, and speech–text alignment features within a discriminative reranking framework. This work is the first to jointly model phoneme-based retrieval and discriminative rescoring without requiring additional labeled data, effectively alleviating the inherent limitation of E2E models in modeling long-tail terms. Evaluated on a benchmark of popular movie titles, our method achieves a 4.4–7.6% relative reduction in word error rate (WER), significantly outperforming multiple strong baselines.

Technology Category

• Natural Language Processing: Speech
• Search and Optimization: Learning to Search
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

End-to-end (E2E) Automatic Speech Recognition (ASR) models are trained using paired audio-text samples that are expensive to obtain, since high-quality ground-truth data requires human annotators. Voice search applications, such as digital media players, leverage ASR to allow users to search by voice as opposed to an on-screen keyboard. However, recent or infrequent movie titles may not be sufficiently represented in the E2E ASR system's training data, and hence, may suffer poor recognition. In this paper, we propose a phonetic correction system that consists of (a) a phonetic search based on the ASR model's output that generates phonetic alternatives that may not be considered by the E2E system, and (b) a rescorer component that combines the ASR model recognition and the phonetic alternatives, and select a final system output. We find that our approach improves word error rate between 4.4 and 7.6% relative on benchmarks of popular movie titles over a series of competitive baselines.

Problem

• Improving ASR accuracy for underrepresented voice search queries
• Addressing poor recognition of recent or infrequent movie titles
• Enhancing phonetic correction and rescoring in E2E ASR systems

Innovation

• Phonetic search generates ASR alternatives
• Rescorer combines ASR and phonetic outputs
• Improves word error rate significantly