Tokenizer Transplantation: Mitigating Autoregressive Collapse in Edge-Efficient Bengali ASR
This work addresses the challenges of word fragmentation and autoregressive collapse in lightweight autoregressive speech recognition models when applied to morphologically rich, non-Latin scripts such as Bengali, primarily caused by English-centric byte-level tokenizers. To overcome this without requiring full model re-pretraining, the authors propose a cross-script lexical transplantation method: replacing the decoder’s byte-level vocabulary with a Bengali-specific WordPiece vocabulary derived from BanglaBERT and rescaling the embedding matrix to better align with the target language’s linguistic structure. This approach enables the first efficient and reproducible tokenizer replacement for compact ASR architectures like Moonshine. Evaluated on the Lipi-Ghor dataset, the method achieves a word error rate of 21.54%, a real-time factor of 0.0053, an 85.8% reduction in autoregressive sequence length, and a substantial drop in word fertility from 9.16 to 1.30.