Search papers, labs, and topics across Lattice.
This paper introduces VocaDet, a novel framework for open-vocabulary object detection and segmentation that leverages user-provided sample collections to learn object concepts without the need for model retraining. By transforming continuous visual representations into discrete visual vocabularies and utilizing a scalable vector database for efficient retrieval, VocaDet significantly enhances the ability to recognize arbitrary objects beyond predefined categories. Experimental results on the UA-DETRAC dataset show that VocaDet achieves effective detection performance while allowing for continuous expansion of its recognition capabilities as more samples are added.
VocaDet enables open-vocabulary object detection that evolves with user input, achieving high performance without retraining the model.
Open-vocabulary object detection and segmentation aim to recognize arbitrary objects beyond predefined categories. Although recent vision-language and reference-based approaches have significantly advanced this field, they often rely on text prompts, limited visual examples, or expensive feature matching procedures, making them difficult to scale to large and continuously expanding object repositories. In this work, we propose VocaDet, a sample-driven open-vocabulary object detection and segmentation framework that learns object concepts directly from user-provided positive and negative sample collections without model retraining. The key idea is to transform continuous visual representations into discrete visual vocabularies and perform efficient retrieval-based recognition through a scalable vector database. Specifically, we employ DINOv3 as the visual feature extractor and apply agglomerative clustering with adaptive clustering sensitivity to generate multi-granularity visual tokens. These visual tokens, together with position-debiased representations and spatial topology information, are stored as expandable object memories in a vector database. During inference, query images are converted into visual tokens and efficiently matched against the stored object memories for object localization and segmentation. Furthermore, a background filtering mechanism is introduced to remove frequently occurring background patterns and reduce redundant retrieval operations in practical fixed-camera scenarios. Experiments on the UA-DETRAC dataset demonstrate that VocaDet achieves effective open-vocabulary detection performance without conventional detector training, while supporting continuously expandable recognition capability as additional positive and negative samples are accumulated.
