CraBERT: Efficient Phoneme Encoder Pre-Training via Cascade Fusion of Subword Representations for Text-to-Speech

2026-06-15

AI Summary

This work addresses the inefficiency and high data demands of phoneme encoder pretraining in text-to-speech synthesis by proposing a cascaded fusion architecture together with a subword–phoneme alignment algorithm. The approach effectively transfers pretrained subword-level BERT representations to the phoneme-level encoder, incorporating lexical and sentential prior knowledge. It achieves, for the first time, efficient integration of subword and phoneme representations, attaining mean opinion scores (MOS) comparable to those of existing methods trained over ten epochs while requiring only approximately one pretraining epoch. This substantially reduces pretraining costs without compromising speech naturalness or prosody, as confirmed by subjective listening evaluations.

Abstract

This paper introduces CraBERT, a pre-trained phoneme encoder (PPEnc) designed for efficient pre-training in text-to-speech (TTS). CraBERT employs a cascade-fusion architecture and a subword-phoneme alignment algorithm to integrate representations from a pre-trained subword-level BERT into a phoneme-level BERT. This design provides prior word- and sentence-level information, reducing the amount of pre-training required by the phoneme encoder. Subjective listening evaluations show that CraBERT achieves MOS values comparable to existing PPEncs after approximately one epoch of pre-training, whereas the baselines in our comparison are pre-trained for approximately ten epochs. These results demonstrate that CraBERT can efficiently learn representations suitable for improving the perceived naturalness and prosody of synthesized speech.

Problem

• text-to-speech
• phoneme encoder
• pre-training efficiency
• speech naturalness
• prosody

Innovation

• cascade fusion
• subword-phoneme alignment
• phoneme encoder
• efficient pre-training
• text-to-speech