Ready to Speak: Aligning LLMs for TTS-Friendly Text Generation

2026-09-01

AI Summary

研究通过偏好对齐方法使大语言模型直接生成适合文本转语音的文本，引入了两个偏好数据集，并使用多种评估方法，发现Feature-aware Sampling and Tuning框架表现最佳。

Abstract

Current Large Language Models (LLMs) are primarily optimized for written text, often producing outputs that are grammatically correct and helpful yet poorly suited for spoken delivery via Text-to-Speech (TTS). In this work, we study how to make LLMs natively generate TTS-friendly text, which we frame as a preference alignment problem: instead of relying on downstream rewriting modules, we directly align LLMs to generate text optimized for spoken delivery. We introduce two preference datasets spanning different target domains, CORA and Recipe, which contain paired TTS-friendly and TTS-unfriendly responses. We further propose an evaluation suite combining a pattern-based heuristic metric, a TTS$\to$ASR evaluation pipeline, and a MUSHRA listening study with human judges. Our experiments compare the recently proposed Feature-aware Sampling and Tuning (FaST) framework -- leveraging interpretable features instead of a black-box reward model -- against an array of alignment baselines on the TTS-friendly generation task. Notably, we found that FaST achieves the best overall tradeoff between TTS-friendliness and helpfulness across various settings. We also identified a strong correlation between our different metrics, highlighting the ability to reliably assess TTS-friendliness via an efficient heuristic.

Problem

• Large Language Models
• Text-to-Speech
• TTS-friendly text
• preference alignment

Innovation

• TTS-friendly text
• Feature-aware Sampling and Tuning (FaST)
• preference alignment