Attention-Based Adversarial Robust Distillation in Radio Signal Classifications for Low-Power IoT Devices
Lightweight Transformers for wireless modulation classification on IoT devices suffer from vulnerability to adversarial attacks—particularly cross-architecture transfer attacks—while struggling to balance robustness and inference efficiency. Method: This paper proposes an attention-mechanism-transfer-based adversarial robust knowledge distillation framework. It distills robust attention maps learned by a large teacher model during adversarial training into a compact Transformer student, without incurring additional inference overhead. The approach integrates adversarial training, attention-guided knowledge distillation, and lightweight architecture design. Contribution/Results: Under FGSM and PGD white-box attacks, the distilled lightweight model achieves significantly enhanced robustness and effectively mitigates adversarial sample transfer across architectures. Experiments demonstrate high-accuracy, robust modulation classification under resource constraints—retaining low latency and power consumption—thus overcoming the critical bottleneck of co-optimizing model lightness and security.