Investigation of Zero-shot Text-to-Speech Models for Enhancing Short-Utterance Speaker Verification

2025-06-17

AI Summary

To address the degradation in accuracy and robustness of speaker verification on short utterances, this paper proposes, for the first time, a zero-shot text-to-speech (ZS-TTS)–driven test-time data augmentation paradigm that requires no model retraining. We systematically evaluate the verification performance gains conferred by speech synthesized from three representative ZS-TTS models—NatureSpeech 3, CosyVoice, and MaskGCT—on the VoxCeleb1 benchmark. Experimental results show that synthetic speech significantly improves short-utterance verification: relative equal error rate (EER) reductions of 10%–16% are achieved across all utterance lengths, with particularly pronounced gains for short segments. Moreover, we observe a saturation effect—diminishing returns beyond a certain utterance duration. This work empirically validates ZS-TTS as a lightweight, plug-and-play augmentation tool, delineating its efficacy and practical limits. It establishes a novel, low-resource-friendly paradigm for speaker verification.

Technology Category

• Natural Language Processing: Speech
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Diffusion Models for Vision

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Short-utterance speaker verification presents significant challenges due to the limited information in brief speech segments, which can undermine accuracy and reliability. Recently, zero-shot text-to-speech (ZS-TTS) systems have made considerable progress in preserving speaker identity. In this study, we explore, for the first time, the use of ZS-TTS systems for test-time data augmentation for speaker verification. We evaluate three state-of-the-art pre-trained ZS-TTS systems, NatureSpeech 3, CosyVoice, and MaskGCT, on the VoxCeleb 1 dataset. Our experimental results show that combining real and synthetic speech samples leads to 10%-16% relative equal error rate (EER) reductions across all durations, with particularly notable improvements for short utterances, all without retraining any existing systems. However, our analysis reveals that longer synthetic speech does not yield the same benefits as longer real speech in reducing EERs. These findings highlight the potential and challenges of using ZS-TTS for test-time speaker verification, offering insights for future research.

Problem

• Enhancing speaker verification with short utterances
• Exploring zero-shot TTS for test-time data augmentation
• Evaluating synthetic speech impact on verification accuracy

Innovation

• Zero-shot TTS for test-time data augmentation
• Combine real and synthetic speech samples
• Evaluate three pre-trained ZS-TTS systems