Search papers, labs, and topics across Lattice.
This paper introduces HOI-DETR, an innovative framework that integrates hand-object and object-object interactions into the Co-DETR architecture, achieving state-of-the-art performance in Hand-Object Interaction (HOI) detection. The authors also present a comprehensive evaluation suite comprising four diverse datasets, including a newly annotated video benchmark from HD-EPIC that enhances the existing Hands23 benchmark. Notably, their trained model demonstrates significant improvements, achieving over 20 percentage points in mean Average Precision (mAP) on the Hands23 and FineBio datasets, underscoring the effectiveness of their approach.
HOI-DETR achieves over 20 percentage points improvement in mAP for Hand-Object Interaction detection, setting a new benchmark in the field.
Understanding hands and the objects they interact with, both directly and through tools, is a key step for tasks ranging from action perception to 3D reconstruction and robotics. Our paper provides several contributions to the Hand-Object Interaction (HOI) understanding literature: (1) HOI-DETR, a new framework that introduces hand-object and object-object interactions to the Co-DETR architecture to produce a state-of-the-art method; (2) a comprehensive HOI evaluation suite of 4 diverse datasets, including a video benchmark derived from the HD-EPIC dataset and fresh annotations that improve the Hands23 benchmark and (3) a trained checkpoint that significantly improves the state of the art across Hands23, HOIST, FineBio, and HD-EPIC, including mAP gains of over 20 percentage points on Hands23 and FineBio. Our ablations confirm the contributions of each model component.
