BranchShine: Compact Raw-Audio-to-IPA Transcription with a RoPE E-Branchformer Encoder

2026-06-22

AI Summary

This work proposes BranchShine, a lightweight end-to-end model for multilingual speech-to-IPA transcription that operates directly on raw audio and comprises only 33 million parameters. The architecture features a compact convolutional frontend followed by a 19-layer E-Branchformer encoder enhanced with rotary position embeddings (RoPE), trained with CTC loss. Evaluated on a diverse test set of 16,660 utterances spanning 41 languages, BranchShine achieves a space-insensitive IPA character error rate of 9.19%, outperforming the 575-million-parameter PhoneticXEUS baseline (9.78%). This demonstrates that substantial model compression can be achieved without sacrificing performance, offering an efficient and compact solution for character-level IPA transcription.

Abstract

Speech-to-IPA transcription is useful when the desired output is pronunciation rather than orthographic text, but competitive multilingual systems are often large and evaluation is sensitive to normalization choices. This paper presents BranchShine, a 33M-parameter raw-audio CTC recognizer with a lightweight convolutional front end and a 19-block RoPE E-Branchformer encoder. We find that BranchShine provides a compact and competitive operating point for IPA transcription under matched normalization and scoring. On a 16,660-utterance multilingual test set covering 41 language labels, BranchShine obtains 9.19% whitespace-insensitive IPA character error rate, compared with 9.78% for the 575.00M-parameter PhoneticXEUS baseline. A secondary child speech reading analysis shows a complementary operating profile: BranchShine is more conservative on incorrect readings, while Whisper-Medium is stronger on exact acceptance of correct readings. Overall, the results indicate that a compact raw-audio-to-IPA model can approach much larger baselines on character-level IPA transcription.

Problem

• speech-to-IPA transcription
• multilingual
• compact model
• normalization sensitivity
• character error rate

Innovation

• raw-audio-to-IPA
• RoPE E-Branchformer
• compact CTC recognizer
• multilingual speech transcription
• character-level IPA error rate