Tone-Conditioned Curriculum Learning for Low-Resource Bantu Speech Recognition

2026-06-30

AI Summary

Existing foundation models exhibit zero-shot word error rates (WER) exceeding 100% on Southern Bantu speech recognition tasks, rendering them impractical. This work proposes a tone-conditioned curriculum learning framework that, for the first time, integrates tonal information into the curriculum mechanism. By combining mixed-difficulty scoring, tone-statistics-gated adapters, and a staged training strategy, the approach significantly enhances cross-lingual transfer performance for low-resource Bantu languages. Experiments on W2V-BERT and Whisper demonstrate that the method reduces average WER to 28.41% across six languages, achieving 23.79% on the Xitsonga transfer task. The study further uncovers systematic interactions between model architecture and language family, offering an effective adaptive training paradigm for low-resource speech recognition.

Abstract

Southern Bantu languages are spoken by over 80 million people, yet current foundation ASR models still produce zero-shot WER above 100%, which limits practical use in education and public services. We addressed this gap with a tone conditioned curriculum framework for 6 Southern Bantu languages that combined hybrid difficulty scoring, gated adapters driven by tonal statistics and staged curriculum training. We trained on a community corpus and tested transfer to NCHLT to measure robustness beyond matched evaluation. Results revealed clear interactions between architecture and language, with W2V-BERT outperforming Whisper on Nguni languages by 3 to 4 WER points whilst Whisper performed better on Sotho-Tswana languages. W2V-BERT with tone conditioning reached 28.41% average WER across datasets and 23.79% on Xitsonga transfer. No single model suited all 6 languages, so deployment should pair model selection per language with validation across corpora.

Problem

• low-resource speech recognition
• Bantu languages
• tone conditioning
• automatic speech recognition
• zero-shot WER

Innovation

• tone-conditioned learning
• curriculum learning
• low-resource ASR
• gated adapters
• Bantu languages