Search papers, labs, and topics across Lattice.
This paper introduces Robust CurveMoE, a novel mixture-of-experts framework designed to enhance the adversarial robustness of neural networks against perturbations defined by varying norm constraints. By connecting models specialized for different norms through low-loss paths and selectively expertizing influential layers, the approach significantly reduces training costs while improving robustness. Experimental results demonstrate that Robust CurveMoE outperforms existing methods, achieving notable increases in Union accuracy on CIFAR-100 and ImageNet-100 datasets.
Achieving a 2.37% boost in Union accuracy over the best baseline, Robust CurveMoE reveals a new paradigm for balancing adversarial defenses across multiple norm constraints.
Multi-norm adversarial defense aims to protect neural networks against perturbations defined by different norm constraints, but existing methods typically optimize competing robustness objectives within a single parameter configuration, leading to substantial training cost and unfavorable robustness trade-offs. We propose Robust CurveMoE, an efficient mixture-of-experts framework that connects models specialized for different perturbation norms through a low-loss path and exploits the complementary robustness profiles of models along this path. Robust CurveMoE derives clean and norm-specialized experts from robustness-constrained curve locations and selectively expertizes only influential layers, while sharing the remaining parameters across routing paths. To further reduce curve-construction cost, we introduce contribution-guided partial updating, which selects influential curve parameters using initialization-based gradient scores. We also theoretically bound the objective gap between partial and full curve optimization. Experiments on CIFAR-100 and ImageNet-100 with WideResNet and Vision Transformer architectures show that Robust CurveMoE consistently improves clean, norm-specific, and Union accuracy over MSD and ERMC. In particular, it improves Union accuracy by 2.37 and 2.13 percentage points over the strongest baseline on CIFAR-100 and ImageNet-100, respectively. Extensive ablations further validate the effectiveness of partial updating, selective expertization, and robustness-constrained expert selection.
