Summary on The Multilingual Conversational Speech Language Model Challenge: Datasets, Tasks, Baselines, and Methods

2025-09-17

AI Summary

This study addresses the challenge of constructing multilingual conversational speech-language models (SLLMs). Method: We organize the first benchmark competition dedicated to real-world multilingual conversational speech-language modeling, releasing a high-quality multilingual dialogue dataset spanning 10+ languages and 1,604 hours of authentic speech; defining two core tasks—end-to-end speech-language modeling and cross-lingual speech dialogue generation; and establishing a unified evaluation framework with strong baseline systems. Contribution/Results: The competition attracted 78 teams globally, yielding 489 valid submissions and 14 technical reports from teams across 13 countries. It establishes the first systematic benchmark for multilingual conversational SLLMs, fostering deep integration of automatic speech recognition, text-to-speech synthesis, cross-lingual transfer learning, and pre-trained speech models, and delivers a reproducible best-practice guide for the community.

Technology Category

• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

This paper summarizes the Interspeech2025 Multilingual Conversational Speech Language Model (MLC-SLM) challenge, which aims to advance the exploration of building effective multilingual conversational speech LLMs (SLLMs). We provide a detailed description of the task settings for the MLC-SLM challenge, the released real-world multilingual conversational speech dataset totaling approximately 1,604 hours, and the baseline systems for participants. The MLC-SLM challenge attracts 78 teams from 13 countries to participate, with 489 valid leaderboard results and 14 technical reports for the two tasks. We distill valuable insights on building multilingual conversational SLLMs based on submissions from participants, aiming to contribute to the advancement of the community.

Problem

• Advancing multilingual conversational speech language models
• Providing task settings and multilingual speech dataset
• Establishing baseline systems for model evaluation

Innovation

• Multilingual conversational speech dataset
• Baseline systems for participants
• Insights from 489 leaderboard results