Voice Conversion for Lombard Speaking Style with Implicit and Explicit Acoustic Feature Conditioning

2025-07-12

AI Summary

This work addresses the challenge of voice conversion in Lombard speech scenarios where target speakers lack Lombard-style recordings. We propose a speaker identity conversion method that operates without any Lombard utterances from the target speaker. Our core innovation is an implicit acoustic feature conditioning strategy: instead of relying on explicit target-speaker Lombard features, the conversion is guided by universal Lombard acoustic priors—such as elevated F1/F2 formants and increased intensity—which preserve critical Lombard characteristics while maintaining high speaker similarity. Experiments demonstrate that our approach achieves intelligibility gains comparable to explicit conditioning models requiring target Lombard data, and significantly outperforms style-agnostic baselines. To the best of our knowledge, this is the first method to enable high-fidelity, zero-shot Lombard style transfer in speaker conversion.

Technology Category

• Natural Language Processing: Speech
• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Text-to-Speech (TTS) systems in Lombard speaking style can improve the overall intelligibility of speech, useful for hearing loss and noisy conditions. However, training those models requires a large amount of data and the Lombard effect is challenging to record due to speaker and noise variability and tiring recording conditions. Voice conversion (VC) has been shown to be a useful augmentation technique to train TTS systems in the absence of recorded data from the target speaker in the target speaking style. In this paper, we are concerned with Lombard speaking style transfer. Our goal is to convert speaker identity while preserving the acoustic attributes that define the Lombard speaking style. We compare voice conversion models with implicit and explicit acoustic feature conditioning. We observe that our proposed implicit conditioning strategy achieves an intelligibility gain comparable to the model conditioned on explicit acoustic features, while also preserving speaker similarity.

Problem

• Transferring Lombard speaking style while preserving speaker identity
• Comparing implicit and explicit acoustic feature conditioning methods
• Improving intelligibility without extensive Lombard effect recordings

Innovation

• Implicit acoustic feature conditioning for VC
• Explicit acoustic feature conditioning comparison
• Lombard style transfer with speaker preservation