TidyVoice 2026 Challenge Evaluation Plan

2026-01-29

AI Summary

This work addresses the significant performance degradation of existing speaker verification systems in cross-lingual scenarios, primarily due to their reliance on English-centric training data. To mitigate this issue, the authors introduce TidyVoiceX, a multilingual dataset encompassing approximately 40 languages derived from Mozilla Common Voice, specifically curated to isolate the impact of language switching on speaker recognition. The study organizes a cross-lingual speaker verification challenge, providing an open-source baseline system and a standardized evaluation protocol, with cross-lingual equal error rate (EER) as the primary metric. This effort establishes the first large-scale, linguistically balanced benchmark for speaker verification, aiming to advance the development of language-agnostic, robust, and equitable speaker recognition technologies.

Technology Category

• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Language and Vision

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

The performance of speaker verification systems degrades significantly under language mismatch, a critical challenge exacerbated by the field's reliance on English-centric data. To address this, we propose the TidyVoice Challenge for cross-lingual speaker verification. The challenge leverages the TidyVoiceX dataset from the novel TidyVoice benchmark, a large-scale, multilingual corpus derived from Mozilla Common Voice, and specifically curated to isolate the effect of language switching across approximately 40 languages. Participants will be tasked with building systems robust to this mismatch, with performance primarily evaluated using the Equal Error Rate on cross-language trials. By providing standardized data, open-source baselines, and a rigorous evaluation protocol, this challenge aims to drive research towards fairer, more inclusive, and language-independent speaker recognition technologies, directly aligning with the Interspeech 2026 theme,"Speaking Together."

Problem

• speaker verification
• language mismatch
• cross-lingual
• multilingual
• language-independent

Innovation

• cross-lingual speaker verification
• TidyVoiceX dataset
• language mismatch
• Equal Error Rate
• multilingual corpus