Hard to Be Heard: Phoneme-Level ASR Analysis of Phonologically Complex, Low-Resource Endangered Languages

2026-04-20

AI Summary

This study addresses the challenge of automatic speech recognition (ASR) for low-resource, phonologically complex endangered languages, where it is difficult to disentangle whether performance bottlenecks stem from data scarcity or linguistic complexity. Focusing on Archi and Rutul—two Northeast Caucasian languages—the authors construct standardized speech–text datasets and introduce a phoneme-level error analysis framework. Their findings reveal an S-shaped learning curve between recognition accuracy and phoneme frequency, suggesting that model generalization can partially overcome data sparsity. By designing language-specific phonemic vocabularies and heuristic output layer initialization for wav2vec2, and benchmarking against state-of-the-art models such as Whisper and Qwen2-Audio, they demonstrate that the optimized wav2vec2 matches or even surpasses Whisper under extremely low-resource conditions. The results indicate that recognition errors are primarily attributable to data scarcity rather than phonological complexity.

Technology Category

• Natural Language Processing: Speech
• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

We present a phoneme-level analysis of automatic speech recognition (ASR) for two low-resourced and phonologically complex East Caucasian languages, Archi and Rutul, based on curated and standardized speech-transcript resources totaling approximately 50 minutes and 1 hour 20 minutes of audio, respectively. Existing recordings and transcriptions are consolidated and processed into a form suitable for ASR training and evaluation. We evaluate several state-of-the-art audio and audio-language models, including wav2vec2, Whisper, and Qwen2-Audio. For wav2vec2, we introduce a language-specific phoneme vocabulary with heuristic output-layer initialization, which yields consistent improvements and achieves performance comparable to or exceeding Whisper in these extremely low-resource settings. Beyond standard word and character error rates, we conduct a detailed phoneme-level error analysis. We find that phoneme recognition accuracy strongly correlates with training frequency, exhibiting a characteristic sigmoid-shaped learning curve. For Archi, this relationship partially breaks for Whisper, pointing to model-specific generalization effects beyond what is predicted by training frequency. Overall, our results indicate that many errors attributed to phonological complexity are better explained by data scarcity. These findings demonstrate the value of phoneme-level evaluation for understanding ASR behavior in low-resource, typologically complex languages.

Problem

• low-resource languages
• phonologically complex languages
• endangered languages
• phoneme-level ASR
• automatic speech recognition

Innovation

• phoneme-level ASR
• low-resource languages
• heuristic output-layer initialization
• phonological complexity
• speech recognition error analysis