Search papers, labs, and topics across Lattice.
This study conducts a detailed empirical analysis of Open Vocabulary Object Detectors (OV-ODs) under various synthetic noise conditions, systematically dissecting their robustness layer-by-layer. The research reveals that robustness is primarily influenced by the image domain rather than annotation strategies, with similar vision backbones exhibiting comparable robustness due to feature collapse at analogous layers. Key improvements in robustness on real-world datasets such as BDD100K and WiderFace are achieved using a lightweight plug-and-play approach that requires significantly fewer trainable parameters compared to traditional end-to-end training methods.
Robustness in Open Vocabulary Object Detectors is driven more by image domain characteristics than by annotation methods, challenging existing assumptions about model training.
The impact of real-world noise on Open Vocabulary Object Detectors (OV-ODs) remains poorly understood due to their architectural complexity. We present our comprehensive analysis Robust Onion, an empirical study that uses controlled synthetic visual degradations to peel OV-ODs layer-by-layer, revealing how, why, and where robustness degrades, systematically analyzing feature collapse. Our findings reveal that models with similar vision backbones exhibit comparable robustness, driven by similar feature collapse at similar layers, while factors such as pretraining strategy, architectural nuances, and caption supervision contribute little. Robustness is primarily governed by the image domain rather than annotations, explaining the similar robustness impact on COCO and LVIS, and why datasets like ODinW-13 can give an impression of inflated robustness due to large, isolated objects. Finally, we validate our insights by improving robustness on real-world BDD100K, WiderFace, and VisDRONE via our lightweight plug-and-play NN & TK0 approach, using 96x fewer trainable parameters than end-to-end training. We also explain the prior works' robustness observations.
