Search papers, labs, and topics across Lattice.
This paper introduces Robust Mixture of Low-Rank Experts (RoME), a novel approach to multi-perturbation adversarial training that enhances robustness against various $\ell_p$ perturbations by utilizing a mixture of experts to create threat-specific pathways. By addressing the limitations of traditional MoE methods, such as threat-agnostic routing and feature redundancy, RoME employs dual-scale gating and threat-guided gating diversification to optimize expert utilization for distinct adversarial threats. Experimental results show that RoME significantly surpasses existing state-of-the-art methods in both union robustness and natural accuracy, while also improving resilience to unseen adversarial threats.
RoME achieves superior robustness against multiple adversarial threats by intelligently routing them through specialized expert pathways, outperforming existing methods in both accuracy and resilience.
Multi-perturbation adversarial training (MAT) aims to achieve robustness against multiple $\ell_p$ perturbations but suffers from robustness trade-offs between different threats. To address this, we employ a mixture of experts (MoE) to route different threats through distinct model pathways. However, naive application of MoE encounters two critical challenges: experts tend to overlook threat-specific features and redundantly capture features shared across threats, and gating networks suffer from threat-agnostic routing where they learn nearly identical routing patterns across threats, thus preventing the construction of threat-specific model pathways. To this end, we propose Robust Mixture of Low-Rank Experts (RoME), where each expert is a low-rank additive update to the shared backbone, allowing it to capture threat-common features while experts focus on threat-specific information. To address threat-agnostic routing, RoME introduces (i) dual-scale gating that exploits threat-discriminative signals from local and global level features, and (ii) threat-guided gating diversification that enforces diverse expert utilization across threats. Extensive experiments demonstrate that RoME outperforms existing state-of-the-art MAT in union robustness and natural accuracy and improves robustness against unseen threats. Codes are available at https://github.com/wkim97/RoME.
