A Fusion-Aware Two-Stage Framework for Mispronunciation Detection and Diagnosis in Low-Resource Modern Standard Arabic

2026-06-22

AI Summary

This work addresses the challenges of data scarcity and domain mismatch between synthetic and real speech in mispronunciation detection (MDD) for Modern Standard Arabic. The authors propose a perception-aware, two-stage end-to-end framework: first learning a universal pronunciation mapping using native and synthetic data, then performing domain adaptation with limited authentic learner recordings to mitigate distributional shift. The approach integrates a pretrained encoder, a causal dilated temporal convolutional network, multi-checkpoint ensemble inference, and N-gram rescoring to enhance prediction stability while avoiding over-correction. Evaluated on the QuranMB.v2 test set, the method achieves an F1 score of 0.7201—representing a 63.1% relative improvement over the baseline—and ranks first in the IqraEval.2 challenge, establishing a new state of the art for low-resource Arabic MDD.

Abstract

Accurate phoneme recognition is pivotal for mispronunciation detection and diagnosis (MDD) in modern standard Arabic (MSA), yet remains constrained by data scarcity and the synthetic-real domain gap. This work proposes a two-stage end-to-end framework. It integrates a pre-trained encoder with causal dilated temporal convolutional networks to preserve fine-grained phonetic variations. A hierarchical two-stage strategy first learns general mappings from native/synthetic corpora, then adapts to scarce real learner data to mitigate domain shift without over-correction. Prediction stability is further enhanced via multi-checkpoint ensemble inference with N-gram rescoring. Evaluated on the QuranMB.v2 test set, our system achieves an F1-score of $0.7201$, a $63.1$\% relative improvement over baseline ($0.4414$). This performance ranks at the top of the IqraEval.2 Challenge, establishing a new state-of-the-art for low-resource MSA in MDD.

Problem

• mispronunciation detection
• low-resource
• Modern Standard Arabic
• domain gap
• phoneme recognition

Innovation

• two-stage framework
• causal dilated temporal convolutional networks
• domain adaptation
• multi-checkpoint ensemble
• mispronunciation detection and diagnosis