Search papers, labs, and topics across Lattice.
This study investigates the handling of fallback situations in a smartwatch-based voice assistant, analyzing over 3,000 real-world user interactions to identify common causes of failures such as transcription errors and ambiguous requests. By establishing an operational taxonomy and creating the annotated VoxFallbacks dataset, the authors provide insights into effective fallback mechanisms that enhance user experience. Key findings reveal that lightweight embedding-based classifiers significantly outperform larger generative models in classification tasks while being more resource-efficient, highlighting a practical approach to improving voice assistant reliability.
Lightweight classifiers can outperform larger models in handling voice assistant fallbacks, transforming user interactions from failures to opportunities.
Robust understanding of user input is a core requirement for voice assistants deployed in real-world environments. In practice, these systems encounter heterogeneous fallback situations caused by noisy audio input, transcription errors, ambiguous requests, incomplete utterances, or unintended activations. Existing systems typically respond with generic fallback messages, which do not resolve the underlying interaction failure and can degrade user experience. We study fallback handling in a deployed smartwatch-based voice assistant for general health support in everyday environments. Our analysis is based on six months of real-world usage data from more than 500 users, yielding a dataset of 3,030 anonymized, naturally occurring fallback-triggering utterances. We contribute (1) an operational taxonomy and the annotated VoxFallbacks dataset of these interactions, (2) a comparative evaluation of different models within a classification pipeline under practical deployment constraints, and (3) practical lessons for designing robust and cost-efficient fallback mechanisms. Results show that lightweight embedding-based classifiers outperform larger generative models on most classification tasks while requiring substantially fewer computational resources.
