Search papers, labs, and topics across Lattice.
This paper introduces REPAIR-Bench, a comprehensive benchmark designed to evaluate robot error perception and recovery in human-robot interactions, based on 214 trials involving 41 participants. The benchmark addresses limitations in prior research by incorporating interdependent failure detection, visual classification of failure types, and user-centered recovery prediction, moving beyond binary success/failure models. Key results demonstrate that hierarchical recurrent modeling significantly enhances failure detection and recovery prediction, achieving a strict F1 score of 0.80 and a Hit@5 of 0.76, thereby providing a valuable resource for advancing adaptive robot systems.
REPAIR-Bench reveals that understanding user recovery strategies can dramatically improve robot interaction reliability and adaptability.
Understanding how users perceive and respond to robot failures is essential for building robust and trustworthy robot systems. Prior work, however, (i) often treats failures as independent events, (ii) emphasizes binary failure detection, (iii) with rule-based recovery modeling. We present REPAIR-Bench, built on 214 interaction trials from 41 participants, the benchmark spans four induced failure types and provides synchronized facial action units, head pose, speech transcripts, and post-interaction affect and recovery reports. The benchmark spans three novel evaluation tasks that jointly capture the lifecycle of failure in human-robot interaction (HRI): (i) failure detection over inter-dependent interaction sessions, modeling longitudinal user adaptation across repeated failures; (ii) visual failure-type classification beyond binary success/failure formulations; and (iii) user-centered recovery prediction, inferring users'preferred recovery strategies from interaction context rather than relying on manually designed or rule-based strategies. In baseline experiments, hierarchical recurrent modeling improved failure detection over a single-session model (strict F1: 0.80 vs. 0.68), achieved a failure localization mean signed error of -0.51 s, median absolute error of 2.97 s and, for recovery prediction, a QLoRA-tuned Mistral-7B reached Hit@5=0.76 and F1@5=0.32. REPAIR-Bench provides both the HRI and Medical HRI communities with a standardized framework for (1) evaluating robot failures and (2) building transparent, adaptive, and trustworthy recovery systems.
