Automatically assessing oral narratives of Afrikaans and isiXhosa children

2025-07-17

AI Summary

This study addresses the high cost and limited scalability of manual assessment of oral narrative competence among preschool children in multilingual South African classrooms. We propose the first automated narrative assessment framework tailored to Afrikaans- and isiXhosa-speaking children. Our method integrates automatic speech recognition (ASR) to transcribe children’s oral narratives, followed by quality and comprehension scoring using both linear models and large language models (LLMs); notably, LLMs are applied for the first time to child language assessment in these indigenous languages. Experimental results demonstrate that the LLM-based system achieves near-expert human accuracy in scoring and significantly outperforms traditional linear models in identifying children requiring instructional intervention. This work establishes a scalable, culturally grounded paradigm for automated assessment in resource-constrained, multilingual educational settings.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Affective Computing

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Developing narrative and comprehension skills in early childhood is critical for later literacy. However, teachers in large preschool classrooms struggle to accurately identify students who require intervention. We present a system for automatically assessing oral narratives of preschool children in Afrikaans and isiXhosa. The system uses automatic speech recognition followed by a machine learning scoring model to predict narrative and comprehension scores. For scoring predicted transcripts, we compare a linear model to a large language model (LLM). The LLM-based system outperforms the linear model in most cases, but the linear system is competitive despite its simplicity. The LLM-based system is comparable to a human expert in flagging children who require intervention. We lay the foundation for automatic oral assessments in classrooms, giving teachers extra capacity to focus on personalised support for children's learning.

Problem

• Automatically assess oral narratives of Afrikaans and isiXhosa children
• Identify preschool children needing intervention via speech recognition
• Compare linear and LLM models for narrative scoring accuracy

Innovation

• Uses automatic speech recognition for narratives
• Compares linear model and LLM for scoring
• LLM-based system matches human expert performance