Bagpiper-TTS: Natural Language Guided Universal Speech Synthesis

2026-06-21

AI Summary

This work proposes Bagpiper-TTS, a unified text-to-speech (TTS) framework that is the first to be driven entirely by general natural language instructions. Addressing the limitations of conventional TTS systems—which rely on fixed input formats and predefined metadata—Bagpiper-TTS integrates natural language understanding, intent reasoning, and speech synthesis to parse user instructions into a structured speech blueprint containing both textual content and fine-grained metadata. This approach enables flexible support for diverse tasks such as multi-speaker synthesis, role-playing, and singing voice generation. The method substantially enhances input flexibility and task generalization, achieving a word error rate of 1.7% on the Seed-TTS-Eval benchmark and matching or surpassing specialized models across multiple applications, with consistent validation from both large-model evaluations and human subjective assessments.

Abstract

Classical TTS systems typically rely on rigid input formats and predefined metadata slots, limiting their ability to fulfill flexible user requirements. This paper introduces Bagpiper-TTS, a universal speech synthesis system that deals with diverse natural language user requests. Given a natural language prompt, Bagpiper-TTS first reasons over the users' intent to derive a rich caption, i.e., a comprehensive textual blueprint encompassing both transcription and nuanced metadata. Subsequently, this caption guides the synthesis of the target speech. Our model inherently supports a broad spectrum of tasks besides classical TTS applications, including multi-talker, intent-to-speech, role-play synthesis, singing voice synthesis, and more. Experimental results demonstrate that Bagpiper-TTS achieves an 1.7% Word Error Rate (WER) on the Seed-TTS-Eval benchmark and match the performance of dedicated models in both LLM-as-a-judge and human subjective evaluations across multiple applications.

Problem

• text-to-speech
• natural language understanding
• speech synthesis
• user intent
• metadata

Innovation

• natural language guided
• universal speech synthesis
• intent reasoning
• rich caption
• multi-task TTS