BlueMagpie-TTS: A Token-Efficient Tokenizer, Language Model, and TTS for Taiwanese-Accent Code-Switching Speech

2026-07-07

AI Summary

This work addresses the suboptimal performance of existing text-to-speech (TTS) systems in synthesizing Taiwan-accented Mandarin and code-switched Mandarin–English speech, which often suffers from accent mismatch, overly fragmented tokenization, and degraded pronunciation at code-switching boundaries. To tackle these issues, the authors propose PangolinTokenizer, a low-overhead byte-level Byte Pair Encoding (BPE) tokenizer tailored for the Taiwan linguistic context, integrated with Barbet—a billion-parameter language model trained on Traditional Chinese—via a learnable bridging module into the pretrained acoustic model VoxCPM2. The proposed approach significantly improves synthesis quality, reducing character error rate (CER) from 11.45% to 4.81% and word error rate (WER) from 14.83% to 5.36% on a 1,000-sentence localized test set. In blind listening evaluations, it achieved a 65.6% listener preference rate, demonstrating its effectiveness and practicality.

Abstract

Off-the-shelf TTS systems are poorly adapted to Taiwanese Mandarin. Their accent defaults to other Mandarin variants, their tokenizers over-segment common Taiwanese text, and their pronunciation degrades at code-switching boundaries where Chinese and English alternate within one utterance. These problems share one root: the text side lacks adaptation to the Taiwanese context. We address the text side from the bottom up. PangolinTokenizer, a byte-level BPE tokenizer trained on Taiwan-context data, reaches the lowest token rate (0.485 tokens/character) with the smallest vocabulary among nine tokenizers. Barbet, a billion-parameter Traditional-Chinese language model trained on PangolinTokenizer, serves as the text-semantic frontend and ranks first among comparable public models on a 14-task evaluation. BlueMagpie-TTS attaches Barbet to the pretrained acoustic stack of VoxCPM2 through a learned bridge, keeping the acoustic stack fixed. On a 1000-sentence Taiwan-localized test set, it lowers CER from 11.45% to 4.81% and WER from 14.83% to 5.36%, relative reductions of 58.0% and 63.9%. In a blind listening study on 500 of these sentences with ten listeners, 65.6% of majority votes prefer BlueMagpie-TTS.

Problem

• Taiwanese Mandarin
• code-switching
• text-to-speech
• tokenizer
• accent adaptation

Innovation

• Token-efficient tokenizer
• Taiwanese-accent Mandarin
• Code-switching TTS
• Language model frontend
• Byte-level BPE