Attention-Based Adversarial Robust Distillation in Radio Signal Classifications for Low-Power IoT Devices

2023-02-01

IEEE Internet of Things Journal

AI Summary

Lightweight Transformers for wireless modulation classification on IoT devices suffer from vulnerability to adversarial attacks—particularly cross-architecture transfer attacks—while struggling to balance robustness and inference efficiency. Method: This paper proposes an attention-mechanism-transfer-based adversarial robust knowledge distillation framework. It distills robust attention maps learned by a large teacher model during adversarial training into a compact Transformer student, without incurring additional inference overhead. The approach integrates adversarial training, attention-guided knowledge distillation, and lightweight architecture design. Contribution/Results: Under FGSM and PGD white-box attacks, the distilled lightweight model achieves significantly enhanced robustness and effectively mitigates adversarial sample transfer across architectures. Experiments demonstrate high-accuracy, robust modulation classification under resource constraints—retaining low latency and power consumption—thus overcoming the critical bottleneck of co-optimizing model lightness and security.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Security and Privacy: Large-scale security measurements

Abstract

Due to great success of transformers in many applications, such as natural language processing and computer vision, transformers have been successfully applied in automatic modulation classification. We have shown that transformer-based radio signal classification is vulnerable to imperceptible and carefully crafted attacks called adversarial examples. Therefore, we propose a defense system against adversarial examples in transformer-based modulation classifications. Considering the need for computationally efficient architecture particularly for Internet of Things (IoT)-based applications or operation of devices in an environment where power supply is limited, we propose a compact transformer for modulation classification. The advantages of robust training such as adversarial training in transformers may not be attainable in compact transformers. By demonstrating this, we propose a novel compact transformer that can enhance robustness in the presence of adversarial attacks. The new method is aimed at transferring the adversarial attention map from the robustly trained large transformer to a compact transformer. The proposed method outperforms the state-of-the-art techniques for the considered white-box scenarios, including the fast gradient method and projected gradient descent attacks. We have provided reasoning of the underlying working mechanisms and investigated the transferability of the adversarial examples between different architectures. The proposed method has the potential to protect the transformer from the transferability of adversarial examples.

Problem

• Defend transformer-based radio signal classification against adversarial attacks
• Develop compact transformer for low-power IoT devices
• Transfer adversarial robustness from large to compact transformers

Innovation

• Attention-based adversarial robust distillation
• Compact transformer for IoT modulation classification
• Transfer adversarial attention map for robustness