From A to B to A: Palindromic Zero-Shot Voice Conversion with Non-Parallel Data
This work proposes a zero-shot cross-lingual voice conversion method that operates without parallel speech data. To address the challenges posed by the absence of explicit alignments and multilingual training corpora, the approach leverages WavLM speech representations and constructs synthetic training pairs from non-parallel source and target utterances via k-nearest neighbor retrieval, employing a “synthetic-to-real” supervised learning paradigm. Additionally, it incorporates a speaker loss derived from a pretrained speaker verification model to enhance target speaker consistency. Trained exclusively on English data, the method achieves high naturalness and strong speaker similarity across multiple languages, significantly outperforming existing baselines and demonstrating robust cross-lingual transfer capability and practical applicability.