TaFD: Threat-Aware Frequency Decoupling for Adversarial Robustness against Heterogeneous Attacks

2026-06-16

AI Summary

This work addresses the challenge that existing joint adversarial training methods suffer from negative transfer when defending against heterogeneous threats—such as ℓp-bounded and semantic attacks—struggling to achieve robustness across diverse attack types. The authors propose a two-stage frequency-domain decoupling defense framework, which first reveals the separability of heterogeneous attacks in the frequency domain. By leveraging spectral prototype clustering, the method identifies distinct threat domains and employs a lightweight classifier to predict attack types. Subsequently, it introduces frequency-conditioned convolution and an expert routing mechanism to dynamically allocate samples to specialized expert models, enabling structured parameter isolation that mitigates optimization conflicts. Evaluated on CIFAR-10, CIFAR-100, and Tiny-ImageNet, the approach improves average robust accuracy by approximately 11% over the strongest baseline while maintaining state-of-the-art clean accuracy.

Abstract

Multi-threat robustness remains a fundamental challenge in deep learning. Although joint adversarial training (JAT) is widely adopted, it suffers from negative transfer under heterogeneous threats, particularly between $\ell_p$-bounded and semantic attacks. Through first-order gradient analysis, we formalize this as gradient incompatibility and theoretically establish the necessity of decoupled optimization. We further reveal that these conflicting threats exhibit separable spectral characteristics in the frequency domain. Motivated by this observation, we propose Threat-aware Frequency Decoupling (TaFD), a two-stage defense framework that reformulates JAT as a frequency-domain divide-and-conquer paradigm. TaFD first discovers latent threat domains via unsupervised clustering of attack spectral prototypes and trains a lightweight classifier for inference-time threat domain identification. Conditioned on the prediction, TaFD employs a Frequency-Conditional Convolution that learns threat-domain-specific spectral masks and routes each sample to the corresponding expert, enforcing structural parameter separation and alleviating optimization conflicts. We validate TaFD on three representative image-classification benchmarks (CIFAR-10, CIFAR-100, and Tiny-ImageNet) and on two representative architectures (the convolutional ResNet and the hybrid-transformer MobileViT). Extensive results demonstrate that TaFD achieves more balanced robustness against heterogeneous attacks than existing JAT and frequency-domain baselines, improving average robust accuracy by approximately 11\% over the strongest baseline while maintaining leading clean accuracy.

Problem

• multi-threat robustness
• heterogeneous attacks
• adversarial robustness
• gradient incompatibility
• frequency domain

Innovation

• Threat-Aware Frequency Decoupling
• Adversarial Robustness
• Heterogeneous Attacks
• Frequency-Domain Analysis
• Decoupled Optimization