Search papers, labs, and topics across Lattice.
This paper introduces RegFormer, a relational grounding transformer module for weakly-supervised Human-Object Interaction (HOI) detection that addresses the computational cost and false positives associated with pairwise reasoning. RegFormer uses spatially grounded signals to guide the reasoning process and promote locality-aware interaction learning under image-level supervision. The results demonstrate that RegFormer efficiently learns spatial cues for instance-level interaction reasoning and achieves performance comparable to fully supervised models.
Achieve fully-supervised HOI detection performance with only image-level labels by learning to ground relationships between humans and objects.
Weakly-supervised Human-Object Interaction (HOI) detection is essential for scalable scene understanding, as it learns interactions from only image-level annotations. Due to the lack of localization signals, prior works typically rely on an external object detector to generate candidate pairs and then infer their interactions through pairwise reasoning. However, this framework often struggles to scale due to the substantial computational cost incurred by enumerating numerous instance pairs. In addition, it suffers from false positives arising from non-interactive combinations, which hinder accurate instance-level HOI reasoning. To address these issues, we introduce Relational Grounding Transformer (RegFormer), a versatile interaction recognition module for efficient and accurate HOI reasoning. Under image-level supervision, RegFormer leverages spatially grounded signals as guidance for the reasoning process and promotes locality-aware interaction learning. By learning localized interaction cues, our module distinguishes humans, objects, and their interactions, enabling direct transfer from image-level interaction reasoning to precise and efficient instance-level reasoning without additional training. Our extensive experiments and analyses demonstrate that RegFormer effectively learns spatial cues for instance-level interaction reasoning, operates with high efficiency, and even achieves performance comparable to fully supervised models. Our code is available at https://github.com/mlvlab/RegFormer.
