Search papers, labs, and topics across Lattice.
This paper introduces a Scene-guided Relational Modeling framework for open-vocabulary object detection, addressing the limitations of existing knowledge distillation methods that neglect structured relationships between objects. By leveraging scene graphs to model interactions and spatial arrangements, the framework enhances the detection of novel object categories through a Relation Attention Module and a scene-based textual alignment branch. Experimental results demonstrate significant improvements in average precision for novel categories on COCO and LVIS datasets, outperforming current state-of-the-art methods.
Capturing structured relationships in images can boost open-vocabulary object detection performance by over 10% on novel categories.
Open-vocabulary object detection seeks to identify novel object categories that were not part of the training data. Many knowledge distillation-based approaches have shown promising performance by transferring knowledge from pre-trained vision-language models to object detection. However, these methods often overlook structured, image-specific relationships between objects, such as interactions and spatial arrangements. This oversight can significantly restrict the effectiveness of detecting novel categories. To address this issue, we propose a Scene-guided Relational Modeling detection framework. This framework utilizes scene graphs to capture structured semantic and spatial relationships between candidate regions and their contextual objects. It explicitly models interactions among neighboring regions and incorporates a Relation Attention Module to implicitly amplify the key relational cues extracted from the scene graph. Furthermore, we present a scene-based textual alignment branch that distills category knowledge from captions to guide relational alignment. This approach facilitates a seamless integration of visual relations with semantic information for enhanced detection performance. Comprehensive experiments show that our model achieves superior performance compared to other OVOD methods, improving the AP for novel categories on COCO and LVIS datasets.
