Your Voice Cloning System is Secretly a Voice Anonymizer

2026-08-27

AI Summary

本文提出利用XTTSv2多语言语音克隆模型进行说话人匿名化，通过迭代优化策略平衡隐私与可懂度，无需重新训练即可实现高质量的语音匿名。

Abstract

Speaker anonymization suppresses speaker-identifying attributes from speech while preserving linguistic content and quality. We propose repurposing XTTSv2, a multilingual voice cloning model trained on 27k hours of speech, for speaker anonymization without retraining. Our key insight is that XTTSv2's voice cloning capabilities preserve prosodic structure independently of speaker identity, enabling voice conversion by conditioning on a pseudo-speaker. We introduce an iterative refinement strategy that balances privacy and utility by maximizing a harmonic mean of speaker dissimilarity and intelligibility. Evaluated on seven European languages across CommonVoice and Multilingual LibriSpeech, our system achieves near-optimal privacy (EER $\approx$ 0.49), competitive intelligibility, and substantially better speech quality than dedicated anonymization baselines, while requiring no language-specific training. We release the code here: https://github.com/rm00cr/coqui-tts.

Problem

• speaker anonymization
• voice cloning
• privacy
• intelligibility

Innovation

• speaker anonymization
• voice cloning
• iterative refinement
• privacy and utility balance
• multilingual