Semantic-Aware Interruption Detection in Spoken Dialogue Systems: Benchmark, Metric, and Model

2026-03-25

AI Summary

This work addresses the challenge in current spoken dialogue systems of balancing response latency and robustness in user interruption detection, exacerbated by the absence of real-world benchmarks and comprehensive evaluation metrics. To this end, we introduce SID-Bench, the first semantics-aware interruption detection benchmark grounded in authentic human–machine conversations, and propose a novel evaluation metric—Average Penalty Time (APT)—which jointly accounts for response delay and false alarm costs. Furthermore, we design a large language model–based training paradigm that effectively captures subtle semantic cues indicative of user intent. Experimental results demonstrate that our approach achieves a nearly threefold reduction in APT compared to state-of-the-art baselines, significantly improving both responsiveness and robustness, and establishing a new state of the art on this task.

Technology Category

• Natural Language Processing: Safety and Robustness
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Humans and AI: Intelligent User Interfaces

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Achieving natural full-duplex interaction in spoken dialogue systems (SDS) remains a challenge due to the difficulty of accurately detecting user interruptions. Current solutions are polarized between "trigger-happy" VAD-based methods that misinterpret backchannels and robust end-to-end models that exhibit unacceptable response delays. Moreover, the absence of real-world benchmarks and holistic metrics hinders progress in the field. This paper presents a comprehensive frame-work to overcome these limitations. We first introduce SID-Bench, the first benchmark for semantic-aware interruption detection built entirely from real-world human dialogues. To provide a rigorous assessment of the responsiveness-robustness trade-off, we propose the Average Penalty Time (APT) metric, which assigns a temporal cost to both false alarms and late responses. Building on this framework, we design an LLM-based detection model optimized through a novel training paradigm to capture subtle semantic cues of intent. Experimental results show that our model significantly outperforms mainstream baselines, achieving a nearly threefold reduction in APT. By successfully resolving the long-standing tension between speed and stability, our work establishes a new state-of-the-art for intelligent interruption handling in SDS. To facilitate future research, SID-Bench and the associated code are available at: https://github.com/xkx-hub/SID-bench.

Problem

• spoken dialogue systems
• interruption detection
• full-duplex interaction
• response latency
• robustness

Innovation

• semantic-aware interruption detection
• SID-Bench
• Average Penalty Time
• full-duplex spoken dialogue systems
• LLM-based detection model