Search papers, labs, and topics across Lattice.
The paper introduces HSA-DINO, a parameter-efficient semantic augmentation framework to improve open-vocabulary object detection (OVOD) performance when transferring to downstream tasks with domain shifts. It uses a multi-scale prompt bank to capture hierarchical semantics and a semantic-aware router to dynamically select augmentation strategies during inference. Experiments on OV-COCO and vertical domain datasets demonstrate that HSA-DINO achieves a better balance between domain adaptability and open-vocabulary generalization compared to existing methods.
Domain shift in open-vocabulary object detection is tamed with a parameter-efficient semantic augmentation framework that progressively enriches textual representations from coarse to fine-grained levels.
Open-vocabulary object detection (OVOD) enables models to detect any object category, including unseen ones. Benefiting from large-scale pre-training, existing OVOD methods achieve strong detection performance on general scenarios (e.g., OV-COCO) but suffer severe performance drops when transferred to downstream tasks with substantial domain shifts. This degradation stems from the scarcity and weak semantics of category labels in domain-specific task, as well as the inability of existing models to capture auxiliary semantics beyond coarse-grained category label. To address these issues, we propose HSA-DINO, a parameter-efficient semantic augmentation framework for enhancing open-vocabulary object detection. Specifically, we propose a multi-scale prompt bank that leverages image feature pyramids to capture hierarchical semantics and select domain-specific local semantic prompts, progressively enriching textual representations from coarse to fine-grained levels. Furthermore, we introduce a semantic-aware router that dynamically selects the appropriate semantic augmentation strategy during inference, thereby preventing parameter updates from degrading the generalization ability of the pre-trained OVOD model. We evaluate HSA-DINO on OV-COCO, several vertical domain datasets, and modified benchmark settings. The results show that HSA-DINO performs favorably against previous state-of-the-art methods, achieving a superior trade-off between domain adaptability and open-vocabulary generalization.
