Automatic Text Pronunciation Correlation Generation and Application for Contextual Biasing

2025-01-01

AI Summary

This work addresses the bottleneck in end-to-end automatic speech recognition (E2E-ASR) where contextual biasing relies on manually curated phoneme dictionaries. We propose the first data-driven Automatic Text–Pronunciation Correspondence (ATPC) method, eliminating the need for any predefined phonetic resources. Our approach leverages weakly supervised speech–text pairs and introduces an Iterative Timestamp Estimator (ITSE) to achieve coarse-grained alignment between text and speech. Within an end-to-end framework, we jointly optimize pronunciation modeling and ASR by integrating a speech encoder with an embedding-distance-based similarity metric. Experiments on multidiialectal Chinese demonstrate substantial improvements in contextual biasing performance. Moreover, the method exhibits strong generalization to unwritten or low-resource languages and dialects lacking phoneme inventories. This work establishes a scalable, resource-agnostic paradigm for pronunciation modeling in low-resource ASR.

Technology Category

• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Machine Learning: Active Learning
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Effectively distinguishing the pronunciation correlations between different written texts is a significant issue in linguistic acoustics. Traditionally, such pronunciation correlations are obtained through manually designed pronunciation lexicons. In this paper, we propose a data-driven method to automatically acquire these pronunciation correlations, called automatic text pronunciation correlation (ATPC). The supervision required for this method is consistent with the supervision needed for training end-to-end automatic speech recognition (E2E-ASR) systems, i.e., speech and corresponding text annotations. First, the iteratively-trained timestamp estimator (ITSE) algorithm is employed to align the speech with their corresponding annotated text symbols. Then, a speech encoder is used to convert the speech into speech embeddings. Finally, we compare the speech embeddings distances of different text symbols to obtain ATPC. Experimental results on Mandarin show that ATPC enhances E2E-ASR performance in contextual biasing and holds promise for dialects or languages lacking artificial pronunciation lexicons.

Problem

• Automatic Phonetics
• Dialects and Languages
• Contextual Understanding

Innovation

• Automatic Text-to-Pronunciation Correlation
• End-to-End Automatic Speech Recognition
• Dialectal Language Support