Synthesizing the Lombard Effect: Multi-Level Control of Speech Clarity and Vocal Effort in TTS

2026-06-22

AI Summary

This work addresses the challenge of modeling the Lombard effect—the human tendency to enhance speech clarity in noisy environments or when communicating with hearing-impaired listeners—in text-to-speech (TTS) synthesis. The authors propose a flow-matching-based TTS framework that incorporates pseudo-labels for vocal effort and articulatory precision, enabling continuous and disentangled multi-level control over these dimensions. Notably, the approach introduces, for the first time in TTS, a word-level emphasis mechanism. The method significantly improves acoustic features associated with speech intelligibility and successfully replicates the comprehension gains observed in human Lombard speech under noise conditions. These results establish a new paradigm for generating highly intelligible synthetic speech.

Abstract

Humans tend to speak louder and clearer in challenging environments, such as noisy conditions or when addressing hearingimpaired listeners, which is called Lombard effect. To simulate this behavior in speech synthesis systems, we introduce a flow-matching based text-to-speech (TTS) model trained with vocal effort and articulation pseudo-labels. The proposed model achieves continuous and disentangled control of vocal effort and articulation, while also enabling word-level emphasis for clarifying specific segments of an utterance. Experimental results show that these control mechanisms effectively improve clarityrelated acoustic features. Furthermore, speech-in-noise experiments demonstrate that our model successfully simulates the intelligibility gains of human clear speech in noisy conditions.

Problem

• Lombard effect
• speech clarity
• vocal effort
• text-to-speech
• intelligibility

Innovation

• Lombard effect
• flow-matching TTS
• vocal effort control
• articulation control
• word-level emphasis