Less can be More: What Aspects of Speech Drive End-of-Turn Detection

2026-09-10

AI Summary

研究通过轻量级三模态分类器分析了声学、韵律和语义信号对对话轮次结束检测的影响，发现韵律特征比语义更能有效且快速地识别说话人何时结束讲话。

Abstract

In conversational AI, detecting when a speaker has finished talking is crucial for natural turn taking. While recent work incorporates semantics, the relative contribution of different modalities remains unclear. We present a controlled ablation of acoustic, prosodic, and semantic signals for streaming end of turn detection using a lightweight trimodal classifier. Under identical training conditions, the acoustic prosodic combination achieves the best balance of accuracy and latency, achieving utterance F1 of 0.93 with 7.8% false alarms at 400ms median latency. Adding text increases premature detections without improving performance. Feature space analysis confirms that prosodic features have the strongest class separability, while text representations overlap substantially. These findings suggest that turn-taking is primarily conveyed through intonation and silence patterns rather than semantic completeness, enabling faster and more reliable systems without expensive text inference.

Problem

• end-of-turn detection
• conversational AI
• modalities

Innovation

• end-of-turn detection
• acoustic prosodic signals
• streaming classification
• turn-taking