Generative Annotation for ASR Named Entity Correction

2025-08-28

AI Summary

End-to-end automatic speech recognition (ASR) systems frequently misrecognize domain-specific named entities (NEs), compromising downstream task performance. Existing phoneme-based edit-distance approaches for named entity correction (NEC) struggle to localize errors involving large morphological discrepancies. To address this, we propose an end-to-end NEC framework that jointly leverages acoustic features and generative sequence labeling: first retrieving candidate entities via acoustic similarity, then performing joint error detection and correction using a generative model. Our approach circumvents the limitations of rigid alignment imposed by conventional edit-distance metrics, thereby significantly improving correction accuracy—particularly for NEs with substantial orthographic or morphological divergence. Extensive evaluation on both public and proprietary test sets demonstrates substantial gains in entity-level accuracy. To foster reproducibility and further research, we release our code, datasets, and pre-trained models.

Technology Category

• Natural Language Processing: Speech
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

End-to-end automatic speech recognition systems often fail to transcribe domain-specific named entities, causing catastrophic failures in downstream tasks. Numerous fast and lightweight named entity correction (NEC) models have been proposed in recent years. These models, mainly leveraging phonetic-level edit distance algorithms, have shown impressive performances. However, when the forms of the wrongly-transcribed words(s) and the ground-truth entity are significantly different, these methods often fail to locate the wrongly transcribed words in hypothesis, thus limiting their usage. We propose a novel NEC method that utilizes speech sound features to retrieve candidate entities. With speech sound features and candidate entities, we inovatively design a generative method to annotate entity errors in ASR transcripts and replace the text with correct entities. This method is effective in scenarios of word form difference. We test our method using open-source and self-constructed test sets. The results demonstrate that our NEC method can bring significant improvement to entity accuracy. We will open source our self-constructed test set and training data.

Problem

• Correcting domain-specific named entities in ASR outputs
• Addressing phonetic differences between transcribed and correct entities
• Improving entity accuracy when word forms differ significantly

Innovation

• Uses speech sound features for candidate retrieval
• Generative method annotates entity errors
• Replaces incorrect text with correct entities