POLY-SIM: Polyglot Speaker Identification with Missing Modality Grand Challenge 2026 Evaluation Plan

2026-03-25

AI Summary

This work addresses the dual challenges of modality absence (e.g., visual unavailability) and cross-lingual variation that critically undermine the robustness and generalization of multimodal speaker recognition in real-world scenarios. To this end, we introduce the POLY-SIM 2026 Challenge, which for the first time systematically integrates modality-missing and cross-lingual speaker recognition into a unified evaluation framework, establishing a new benchmark under multilingual conditions with diverse modality-missing settings. Leveraging multimodal fusion, cross-lingual representation learning, and robust modeling techniques for missing modalities, the initiative provides standardized datasets, baseline models, and application-oriented evaluation protocols, thereby laying a foundation for advancing multimodal speaker recognition performance in complex, real-world environments.

Technology Category

• Machine Learning: Multimodal Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Multimodal speaker identification systems typically assume the availability of complete and homogeneous audio-visual modalities during both training and testing. However, in real-world applications, such assumptions often do not hold. Visual information may be missing due to occlusions, camera failures, or privacy constraints, while multilingual speakers introduce additional complexity due to linguistic variability across languages. These challenges significantly affect the robustness and generalization of multimodal speaker identification systems. The POLY-SIM Grand Challenge 2026 aims to advance research in multimodal speaker identification under missing-modality and cross-lingual conditions. Specifically, the Grand Challenge encourages the development of robust methods that can effectively leverage incomplete multimodal inputs while maintaining strong performance across different languages. This report presents the design and organization of the POLY-SIM Grand Challenge 2026, including the dataset, task formulation, evaluation protocol, and baseline model. By providing a standardized benchmark and evaluation framework, the challenge aims to foster progress toward more robust and practical multimodal speaker identification systems.

Problem

• multimodal speaker identification
• missing modality
• cross-lingual
• robustness
• generalization

Innovation

• missing modality
• cross-lingual speaker identification
• multimodal robustness
• polyglot speaker recognition
• grand challenge benchmark