An End-to-End Approach for Child Reading Assessment in the Xhosa Language

2025-05-23

AI Summary

This study addresses the challenge of automating reading proficiency assessment for children speaking Xhosa—a low-resource language—by introducing the first expert-validated (EGRA-aligned), multi-dimensionally annotated speech dataset of Xhosa children’s oral reading. To tackle the dual challenges of complex child-speaker acoustics and severe data scarcity, we systematically evaluate end-to-end speech models—including wav2vec 2.0, HuBERT, and Whisper—on this task. We present the first empirical evidence that multi-class joint training significantly enhances wav2vec 2.0’s performance under few-shot conditions; ablation studies confirm that both dataset scale and class balance critically impact model accuracy. Under joint training, wav2vec 2.0 substantially outperforms single-task baselines. Our work establishes a scalable, low-cost technical framework for automated literacy assessment in low-resource languages and provides a foundational benchmark dataset for future research.

Technology Category

• Natural Language Processing: Speech
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: AI for Accessibility

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Large pretrained models with web data
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Child literacy is a strong predictor of life outcomes at the subsequent stages of an individual's life. This points to a need for targeted interventions in vulnerable low and middle income populations to help bridge the gap between literacy levels in these regions and high income ones. In this effort, reading assessments provide an important tool to measure the effectiveness of these programs and AI can be a reliable and economical tool to support educators with this task. Developing accurate automatic reading assessment systems for child speech in low-resource languages poses significant challenges due to limited data and the unique acoustic properties of children's voices. This study focuses on Xhosa, a language spoken in South Africa, to advance child speech recognition capabilities. We present a novel dataset composed of child speech samples in Xhosa. The dataset is available upon request and contains ten words and letters, which are part of the Early Grade Reading Assessment (EGRA) system. Each recording is labeled with an online and cost-effective approach by multiple markers and a subsample is validated by an independent EGRA reviewer. This dataset is evaluated with three fine-tuned state-of-the-art end-to-end models: wav2vec 2.0, HuBERT, and Whisper. The results indicate that the performance of these models can be significantly influenced by the amount and balancing of the available training data, which is fundamental for cost-effective large dataset collection. Furthermore, our experiments indicate that the wav2vec 2.0 performance is improved by training on multiple classes at a time, even when the number of available samples is constrained.

Problem

• Developing accurate child speech recognition for low-resource Xhosa language
• Addressing limited data and unique acoustic properties in child speech
• Evaluating cost-effective AI models for reading assessment in low-income regions

Innovation

• End-to-end models for Xhosa child speech
• Novel dataset with EGRA-aligned samples
• Multi-class training boosts wav2vec 2.0