Not All Fallbacks Are Failures: Understanding and Recovering from Fallbacks in Mobile Voice Assistants

2026-08-31

AI Summary

研究分析了移动语音助手在实际使用中遇到的多种失败情况，并提出了一种基于轻量级嵌入式分类器的方法来改进系统对这些失败情况的处理，从而提升用户体验。

Abstract

Robust understanding of user input is a core requirement for voice assistants deployed in real-world environments. In practice, these systems encounter heterogeneous fallback situations caused by noisy audio input, transcription errors, ambiguous requests, incomplete utterances, or unintended activations. Existing systems typically respond with generic fallback messages, which do not resolve the underlying interaction failure and can degrade user experience. We study fallback handling in a deployed smartwatch-based voice assistant for general health support in everyday environments. Our analysis is based on six months of real-world usage data from more than 500 users, yielding a dataset of 3,030 anonymized, naturally occurring fallback-triggering utterances. We contribute (1) an operational taxonomy and the annotated VoxFallbacks dataset of these interactions, (2) a comparative evaluation of different models within a classification pipeline under practical deployment constraints, and (3) practical lessons for designing robust and cost-efficient fallback mechanisms. Results show that lightweight embedding-based classifiers outperform larger generative models on most classification tasks while requiring substantially fewer computational resources.

Problem

• fallbacks
• voice assistants
• user experience
• interaction failure
• noisy audio input

Innovation

• fallback handling
• embedding-based classifiers
• real-world usage data