Do Speech Emphasis Models Generalize across Languages and Emotions?

2026-06-26

AI Summary

This study addresses the limited generalizability of existing speech emphasis detection models, which are typically trained on monolingual, neutral-read speech and struggle in real-world multilingual and emotionally diverse settings. To bridge this gap, the authors introduce MMEE, a novel multilingual and multi-emotional emphasis corpus encompassing seven languages and 34 emotion or style categories. They present the first systematic evaluation of emphasis detection models across multidimensional settings, including cross-lingual and cross-emotional generalization. Benchmark experiments based on two state-of-the-art architectures demonstrate that multilingual training substantially enhances model robustness, reveal strong transferability between high- and low-arousal emotional states, and show that competitive performance can be maintained even with limited training data.

Abstract

Prosodic emphasis varies across languages, emotions, and speaking styles, yet existing emphasis detection models are largely trained and evaluated on monolingual neutral read speech. We introduce MMEE (Multilingual Multi-Emotion Emphasis), a corpus of 10,000 professionally recorded expressive utterances (14.13 hours) across 7 languages and 34 emotion/style categories, with three-level perceptual labels (10 annotations per sample). We benchmark two state-of-the-art architectures under monolingual, cross-lingual, multilingual, cross-emotion, cross-dataset, and data-scale settings. Monolingual models show limited zero-shot transfer, degrading across typologically distant languages, while multilingual training substantially improves robustness. Models transfer robustly between high- and low-arousal emotions; bidirectional transfer between synthetic and perceptual benchmarks suggests shared prosodic structure; and performance stays robust even at smaller training scales.

Problem

• speech emphasis
• cross-lingual generalization
• emotion variation
• prosody
• multilingual speech

Innovation

• multilingual speech
• emotion prosody
• emphasis detection
• cross-lingual generalization
• perceptual labeling