Search papers, labs, and topics across Lattice.
This paper introduces Season, a novel framework designed to enhance transfer-based adversarial attacks by addressing the architectural differences between CNNs and Vision Transformers. By decomposing updates into low-frequency structural cues and high-frequency textures, Season effectively mitigates feature interference through orthogonal projection, allowing for improved transfer success rates. The method demonstrates a significant average improvement of 6.6 percentage points in transfer success across various model architectures, with some cases achieving up to 16.0 points over established baselines.
Season redefines adversarial attack strategies by seamlessly integrating structural and textural updates, achieving unprecedented transfer success across diverse model architectures.
Transfer-based adversarial attacks often transfer poorly across heterogeneous architectures because CNNs favor local textures while Vision Transformers (ViTs) rely on global shapes. We propose Season, a spectrum-aware orthogonal gradient refinement framework for L-infinity transfer attacks against black-box target models on ImageNet, using a white-box surrogate. Season decomposes each update into a low-frequency branch capturing structural cues and a high-frequency branch capturing textures. A low-saliency guidance scheme reallocates high-frequency energy to background regions, preserving foreground structures that ViTs depend on. An orthogonal projection then forces the textural update to lie in the orthogonal complement of the structural direction, mitigating feature interference. As a training-free plug-and-play wrapper, Season enhances eight gradient-stabilization and input-enhancement attacks without modifying their cores. Across eight CNN, ViT, and MLP targets, Season improves transfer success rate by 6.6 percentage points on average and up to 16.0 points over strong baselines under a unified protocol.
