VoxSumm: A Multilingual Corpus of Long-Form Spoken News for Joint Summarization and Translation

2026-08-10

AI Summary

This study addresses the gap in existing research, which predominantly focuses on text-based summarization for long documents and treats translation separately from compression in multilingual speech processing, thereby neglecting joint modeling for cross-lingual spoken summarization. To bridge this gap, we propose Joint Summarization and Translation (JSumT), the first end-to-end framework that directly generates concise summaries in a target language from long-form source-language speech. We introduce VoxSumm, the first multilingual spoken summarization corpus supporting this task, comprising 703 hours of speech across 24 languages. Experimental results demonstrate that Gemini-1.5-Pro achieves the most consistent performance, with English summaries generally outperforming those in other languages. Furthermore, the conventional “translate-then-summarize” pipeline frequently fails to follow instructions, underscoring the necessity of joint modeling.

Abstract

As information increasingly traverses linguistic boundaries, users require concise cross-lingual representations of long-form content. Nevertheless, long-document summarization research remains text-centric, whereas multilingual speech research has largely prioritized translation, preserving source content rather than compressing it. We address this methodological gap by formalizing joint speech summarization and translation (JSumT): the generation of a succinct, faithful target-language summary directly from a long spoken document in a source language. We additionally introduce VoxSumm, the first multilingual and cross-lingual benchmark for this task, comprising 10,045 BBC article-summary pairs across 24 languages and encompassing approximately 703 hours of speech data. Our evaluation of representative speech-language models reveals pronounced variation across models and generation settings: Gemini3.1-Pro demonstrates the greatest consistency, summarization into English generally surpasses generation into non-English target languages, and translating an entire document before summarization compounds instruction-following failures. Through the release of VoxSumm, we establish a foundation for developing and evaluating multilingual systems capable of jointly interpreting, compressing, and translating long-form speech.

Problem

• joint summarization and translation
• long-form spoken news
• multilingual speech
• cross-lingual summarization
• speech summarization

Innovation

• joint speech summarization and translation
• multilingual spoken corpus
• long-form speech processing
• cross-lingual summarization
• VoxSumm