Search papers, labs, and topics across Lattice.
This study investigates the reliability of confidence scores produced by open-vocabulary object detectors, revealing that these scores are biased mixtures of visual scale and semantic specificity. Through controlled experiments on datasets like COCO and LVIS, the authors quantify the scale bias, which inflates scores for larger objects, and semantic bias, which suppresses scores for generic queries, demonstrating that these biases stem from the inherent structure of CLIP's pretraining. A proposed temperature scaling correction significantly improves detection performance for small objects without retraining, highlighting a critical limitation in adapting image-level models for region-level tasks.
Confidence scores from open-vocabulary detectors are fundamentally flawed, conflating object size and query specificity, leading to misleading localization estimates.
Foundation models such as CLIP have enabled open-vocabulary object detectors that generalise to novel categories via vision-language similarity. However, the confidence scores these detectors produce are not reliable localization probability estimates: they conflate visual scale and semantic query specificity with the true detection signal. Through controlled experiments on COCO across three foundation-model-based detectors (GroundingDINO, OWL-ViT, YOLO-World), with the scale-bias finding further replicated on LVIS (1,203 categories) using GroundingDINO, we show that s=cos(v,t) is a biased mixture of two effects. Scale bias (alpha = +0.064, r = 0.579, p = 1.29 x 10^-58) systematically inflates scores for large objects. Semantic bias (beta = -0.705, p = 5.23 x 10^-41) suppresses scores for generic queries. Both biases are structurally inevitable from CLIP's image-level pretraining. Threshold adjustment cannot remove them: oracle per-scale thresholding yields Delta F1 = +0.001 for small objects versus +0.102 for large. A parameter-free temperature scaling correction improves small-object Recall@10 by 19.6% (p < 0.01) without retraining. This comes at a modest, measurable cost to pooled-ranking precision, so the bias is partially, not freely, reversible at inference time. These findings reveal a fundamental limitation of adapting image-level foundation models to region-level detection tasks.
