RoME: Robust Mixture of Low-Rank Experts against Multiple Adversarial Perturbations

2026-07-07

AI Summary

This work addresses the robustness trade-offs among different ℓ_p threat models in multi-perturbation adversarial training by proposing a novel mixture-of-experts (MoE) architecture. The approach introduces low-rank experts to disentangle shared and perturbation-specific features, complemented by a dual-scale gating mechanism and a threat-guided gating diversification strategy to mitigate expert redundancy and indiscriminate routing. Experimental results demonstrate that the proposed method achieves substantially improved joint robustness against multiple known ℓ_p perturbations while maintaining high natural accuracy. Moreover, it exhibits superior generalization capability to unseen perturbation types, outperforming current state-of-the-art methods in overall performance.

Abstract

Multi-perturbation adversarial training (MAT) aims to achieve robustness against multiple $\ell_p$ perturbations but suffers from robustness trade-offs between different threats. To address this, we employ a mixture of experts (MoE) to route different threats through distinct model pathways. However, naive application of MoE encounters two critical challenges: experts tend to overlook threat-specific features and redundantly capture features shared across threats, and gating networks suffer from threat-agnostic routing where they learn nearly identical routing patterns across threats, thus preventing the construction of threat-specific model pathways. To this end, we propose Robust Mixture of Low-Rank Experts (RoME), where each expert is a low-rank additive update to the shared backbone, allowing it to capture threat-common features while experts focus on threat-specific information. To address threat-agnostic routing, RoME introduces (i) dual-scale gating that exploits threat-discriminative signals from local and global level features, and (ii) threat-guided gating diversification that enforces diverse expert utilization across threats. Extensive experiments demonstrate that RoME outperforms existing state-of-the-art MAT in union robustness and natural accuracy and improves robustness against unseen threats. Codes are available at https://github.com/wkim97/RoME.

Problem

• multi-perturbation adversarial training
• robustness trade-offs
• adversarial perturbations
• mixture of experts
• threat-specific robustness

Innovation

• Mixture of Experts
• Low-Rank Adaptation
• Multi-perturbation Adversarial Training
• Threat-specific Routing
• Robustness Trade-off