🤖 AI Summary
This work addresses the performance gap in multilingual automatic speech recognition for low-resource Central Asian languages—such as Kazakh, Kyrgyz, and Uzbek—caused by severe data scarcity. The authors propose a robust foundation model trained on 2 million hours of audio using a HuBERT-style objective to pretrain a Conformer encoder. To mitigate dominance by high-resource languages, they introduce a novel cluster-level data balancing strategy and a domain-aware fine-tuning sampling approach. Experimental results demonstrate that the proposed model significantly outperforms strong open-source baselines like Whisper Large v3, particularly on spontaneous speech, while maintaining efficient inference capabilities.
📝 Abstract
Despite recent scaling successes, multilingual ASR performance remains highly uneven, with long-tail languages suffering from severe data scarcity. This work addresses the challenge of building robust foundation models for underrepresented Central Asian languages (Kazakh, Kyrgyz, Uzbek). We present GigaAM Multilingual, a Conformer encoder pre-trained on 2M hours of audio using a HuBERT-style objective. Crucially, we introduce a cluster-level data balancing strategy during pre-training and a domain-aware sampling method during fine-tuning to mitigate head-language dominance. In controlled comparisons, our approach outperforms strong open pretrained encoders (Whisper Large v3, Omnilingual-1B) on target languages, achieving significant gains on spontaneous speech while maintaining efficiency. We release the foundation encoder and ASR model, offering a proven recipe for effective multilingual adaptation under realistic data imbalance.