Search papers, labs, and topics across Lattice.
This paper introduces a unified benchmark for evaluating adversarial attacks on object detection models, addressing the lack of standardized evaluation in the field. The benchmark incorporates metrics to differentiate localization and classification errors and assesses attack cost using perceptual metrics. Through extensive experiments, the authors find that modern adversarial attacks exhibit poor transferability from CNNs to Vision Transformers and that adversarial training with a diverse mix of high-perturbation attacks yields the most robust defense.
Transformer-based object detectors are surprisingly resistant to adversarial attacks crafted for CNNs, but can be hardened further by training against a diverse "cocktail" of strong attacks.
Object detection models are critical components of automated systems, such as autonomous vehicles and perception-based robots, but their sensitivity to adversarial attacks poses a serious security risk. Progress in defending these models lags behind classification, hindered by a lack of standardized evaluation. It is nearly impossible to thoroughly compare attack or defense methods, as existing work uses different datasets, inconsistent efficiency metrics, and varied measures of perturbation cost. This paper addresses this gap by investigating three key questions: (1) How can we create a fair benchmark to impartially compare attacks? (2) How well do modern attacks transfer across different architectures, especially from Convolutional Neural Networks to Vision Transformers? (3) What is the most effective adversarial training strategy for robust defense? To answer these, we first propose a unified benchmark framework focused on digital, non-patch-based attacks. This framework introduces specific metrics to disentangle localization and classification errors and evaluates attack cost using multiple perceptual metrics. Using this benchmark, we conduct extensive experiments on state-of-the-art attacks and a wide range of detectors. Our findings reveal two major conclusions: first, modern adversarial attacks against object detection models show a significant lack of transferability to transformer-based architectures. Second, we demonstrate that the most robust adversarial training strategy leverages a dataset composed of a mix of high-perturbation attacks with different objectives (e.g., spatial and semantic), which outperforms training on any single attack.
