BuzzASR: A Swarm of 100+ Monolingual Speech Recognition Models

2026-09-08

AI Summary

为解决多语言模型在小语种上表现不佳的问题，通过针对102种语言进行单语微调及更复杂的语言适应策略，提高了自动语音识别的准确性。

Abstract

We introduce BuzzASR, a collection of language-specialized fine-tuned Whisper models adapted for automatic speech recognition (ASR) in 102 languages. Large end-to-end Transformer-based ASR models such as Whisper have revolutionized ASR, but most prominent models are highly multilingual. As a result, these models often perform poorly on languages less well-represented in their training set. While it has long been known that effective language adaptation can be achieved through simple fine-tuning on monolingual data, this strategy has only been applied to a small number of languages. We massively scale up this simple approach to 102 languages covered in the FLEURS dataset, while also implementing a more complex language adaptation strategy that integrates monolingual tokenizer replacement and data augmentation using text-only fine-tuning. BuzzASR models outperform Whisper-large-v3 on 77 out of 102 languages, reducing character error rates (CER) by a factor of over 2.8 on average. Our models achieve state-of-the-art CER among open-source systems on 27 of 102 languages on the combined FLEURS and Common Voice test set. Our tokenizer replacement strategy yields an average 3.3x improvement in compression rate (characters per token) over Whisper's multilingual BPE, with gains of up to 21.7x. We release all models, code, and detailed results: https://lemn-lab.github.io/buzz-asr

Problem

• Automatic Speech Recognition
• Language Adaptation
• Multilingual Models
• Monolingual Data

Innovation

• monolingual fine-tuning
• language adaptation
• tokenizer replacement
• data augmentation
• compression rate improvement