When Models Hear What They Expect: Diagnosing Prosodic Heuristics in Multimodal Sarcasm Detection

2026-08-30

AI Summary

研究探讨了多模态大语言模型在讽刺检测中是否依赖于语调线索，通过控制实验发现模型误判主要基于高音调和不规则停顿的刻板印象。

Abstract

Multimodal Large Language Models (MLLMs) process speech and text jointly, yet whether they exploit prosodic cues for pragmatic inference or rely on surface acoustic patterns has received little systematic investigation. We address this through sarcasm detection, evaluating Qwen2.5-Omni and Qwen3-Omni on Mandarin Chinese and English under five modality conditions that decompose the contributions of lexical content, vocal semantics, and prosodic structure. Adding audio systematically inflates false positives without improving true positive detection. Acoustic error diagnosis reveals that model errors cluster on a shared stereotype of expressive prosody, namely elevated pitch and irregular pausing, that diverges from the actual cues marking sarcasm in both languages. Targeted manipulation of only these two dimensions causally confirms the heuristic, inducing false positive rates of up to 60%. Applying the same manipulation template to Gemini~3 Flash Preview without modification replicates the effect, suggesting that the stereotype extends beyond the Qwen Omni family rather than arising from a single model architecture.

Problem

• Prosodic Heuristics
• Multimodal Sarcasm Detection
• Acoustic Patterns

Innovation

• prosodic cues
• sarcasm detection
• multimodal large language models
• false positives