Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The ERR@HRI 3.0 Challenge focused on advancing multimodal machine learning methods for detecting and anticipating errors in human-robot interactions using two novel datasets: the Bystander Affect Detection (BAD) dataset and the Bad Idea dataset. These datasets, derived from naturalistic settings, provide raw video data capturing spontaneous and anticipatory human reactions to robot failures, enabling researchers to develop more generalizable error detection systems. Notably, all participating teams exceeded the performance of established convolutional neural network baselines, highlighting the potential for improved context-aware interaction systems in real-world applications.
Teams exceeded baseline performance in detecting human reactions to robot failures, showcasing the power of naturalistic datasets in training robust models.
As robots become increasingly integrated into human environments, their ability to detect and respond to errors remains critical for maintaining user trust and interaction quality. While recent advances in machine learning have improved error detection capabilities, most approaches are limited to specific contexts, controlled settings, or pre-extracted features, limiting their generalizability and applicability to real-world conditions. To address this challenge, the third edition of the ERR@HRI Challenge (ERR@HRI 3.0) provided researchers with two complementary datasets that enable end-to-end innovation in methods for both detecting and preventing errors in human-robot interaction. The challenge offered raw, non-anonymized video data from naturalistic settings: (1) the Bystander Affect Detection (BAD) dataset, containing webcam recordings of 45 participants'spontaneous reactions to robot and human failure scenarios; and (2) the Bad Idea dataset, featuring 29 participants'anticipatory facial responses while predicting action outcomes before failures occur. Both datasets were collected via crowdsourcing, capturing the inherent variability of real-world conditions. This naturalistic variability, while challenging, provides an authentic testbed for developing robust error detection systems. Participants developed multimodal machine learning models for bystander reaction detection (Track 1) and anticipatory outcome prediction (Track 2), with an optional cross-dataset generalization track (Track 3). Three teams submitted valid models, all of which surpassed our convolutional neural network baselines. This paper describes the datasets, tasks, baselines, and results of ERR@HRI 3.0, and discusses implications for building generalizable, context-aware, and anticipatory error detection systems for human-robot interaction.