Search papers, labs, and topics across Lattice.
This paper introduces InfraQR, a novel structured patch attack specifically designed for infrared vision-language models, which have been underexplored in terms of robustness against localized perturbations. By strategically placing a compact, structured patch along image boundaries and optimizing it through surrogate CLIP-style encoders, InfraQR significantly degrades the performance of various classifiers and models, including a drastic drop in OpenAI CLIP accuracy from 98.67% to 0.70%. The findings highlight the vulnerability of infrared vision-language models to structured attacks, emphasizing the need for enhanced security measures in multi-task settings beyond traditional object occlusion strategies.
InfraQR reveals that infrared vision-language models can be drastically misled by structured edge-placed perturbations, with accuracy plummeting from 98.67% to 0.70%.
Infrared vision-language models are increasingly used for perception under low-light and adverse visual conditions, yet their robustness to localized structured perturbations remains underexplored. Existing infrared adversarial studies mainly focus on object detectors, leaving the security of infrared vision-language models less systematically examined. We present InfraQR, a QR-inspired structured patch attack for infrared vision-language models. Unlike localized attacks that attach perturbations to the target object, InfraQR places a compact structured patch along image boundaries and optimizes learnable grid cells through surrogate CLIP-style encoders. The resulting patch has a near-binary structured appearance, but is not required to be a valid or machine-readable QR code. We evaluate InfraQR on infrared classification, caption transfer, and question-answer-aware visual question answering (VQA) tasks. On a 300-image infrared benchmark, InfraQR sharply reduces the accuracy of multiple CLIP-style classifiers, including reducing OpenAI CLIP accuracy from 98.67% to 0.70%. The generated adversarial images also transfer to black-box captioning and VQA models, causing semantic degradation in captions and more error-prone answers under GPT-5.4-based evaluation. These results show that infrared vision-language models remain vulnerable to structured edge-placed perturbations, motivating further study of cross-task robustness beyond direct object occlusion.
