Hybrid Continual Learning for Low-Resource Australian Aboriginal Language Identification

2026-07-11

AI Summary

This work addresses the challenge of catastrophic forgetting when adapting pretrained speech models to Australian Indigenous languages under extreme data scarcity, which severely undermines multilingual continual learning performance. To mitigate this issue, the authors propose a hybrid continual learning framework that integrates replay-augmented elastic weight consolidation with constraint-guided knowledge distillation. Evaluated on three low-resource Indigenous languages—Warlpiri, Dalabon, and Dharawal—the approach significantly outperforms standard fine-tuning and existing continual learning baselines. It simultaneously improves recognition accuracy for new languages while preserving performance on high-resource languages, thereby enabling effective cross-lingual knowledge transfer and long-term retention in extremely low-resource scenarios.

Abstract

Language identification is an important step toward integrating endangered Australian Aboriginal languages (AALs) into speech technologies supporting language revitalisation and digital inclusion. However, extreme data scarcity limits model performance. Transfer learning from high-resource languages shows promise but often suffers from catastrophic forgetting when adapting to new languages. Continual learning (CL) can mitigate this issue, though it remains challenging with very limited data. To address this, we propose two hybrid continual learning methods: Replay Augmented Elastic Weight Consolidation and Constraint Guided Knowledge Distillation to adapt pretrained speech models for AAL identification while preserving previously learned knowledge. Experiments on Warlpiri, Dalabon and Dharawal show that the proposed methods outperform fine-tuning and existing CL baselines, improving adaptation to multiple AALs while maintaining performance on previously learnt high-resource languages.

Problem

• continual learning
• language identification
• low-resource languages
• catastrophic forgetting
• Australian Aboriginal languages

Innovation

• Continual Learning
• Low-Resource Language Identification
• Catastrophic Forgetting
• Knowledge Distillation
• Transfer Learning