Search papers, labs, and topics across Lattice.
This paper introduces MOAT, a model-agnostic defense mechanism designed to protect Vision Transformers (ViTs) from adversarial attacks targeting token pruning techniques. By applying a series of randomized input transformations, MOAT effectively mitigates efficiency degradation while maintaining model performance. Experimental results show that MOAT restricts the degradation of GFLOPs under attack to within 3.4% of the original model, demonstrating its robustness across various ViT architectures.
Adversarial attacks can degrade the efficiency of Vision Transformers, but MOAT ensures that performance remains nearly intact, limiting GFLOPs loss to just 3.4%.
To adopt the Vision Transformers (ViTs) in resource-constrained environment, token pruning is widely used to reduce computational cost without impacting accuracy. However, adversaries have developed targeted attacks against said token pruning techniques to undermine such attempts to make ViTs efficient. In this paper, we propose MOAT, a model-agnostic pre-processing defense pipeline that applies a combination of input transformations to protect efficient ViT implementations against adversarial efficiency attacks. MOAT operates directly on the input without requiring modifications to the model architecture or token pruning mechanism. Experimental results demonstrate that, across all evaluated ViT models, MOAT limits GFLOPs degradation under adversarial attacks to within 3.4% of the original unattacked model.
