$S^3$-Bench: Evaluating Speech Interaction Models as Scientific Voice Assistants

2026-09-09

AI Summary

本文提出S^3-Bench，一个针对科学领域语音交互模型的评估框架，通过知识问答和多轮对话系统地解决专业术语理解和准确响应生成的问题。

Abstract

The advance of multimodal large language models (MLLMs) has fundamentally reshaped the paradigm of human-computer interaction, especially speech interaction models capable of seamless conversations. Despite remarkable performance as general voice assistants, their performance in specialized domains remains underexplored, particularly in scientific areas. Scientific interactions introduce formidable challenges, involving rare technical terminology, spoken norms of abbreviations, and the natural verbalization of symbolic special expressions. In this paper, we introduce S$^3$-Bench, a systematic evaluation framework covering 10 major disciplines, consisting of a Knowledge set for speech question-answering and a Dialogue set for multi-turn progressive interactions with simulated user agents. By decomposing a complete atomic turn into stages of speech recognition, perception, knowledge utilization with reasoning, and response pronunciation, we systematically characterize the common challenges and performance tradeoffs of existing approaches. Furthermore, experiments on multi-turn interactions reveal persistent limitations in user adaptation and the generation of accurate, comprehensive, and efficient responses.

Problem

• multimodal large language models
• speech interaction
• scientific domains

Innovation

• S^3-Bench
• multimodal large language models
• scientific voice assistants
• multi-turn interactions
• performance tradeoffs