Gradient Norm-based Fine-Tuning for Backdoor Defense in Automatic Speech Recognition

2025-02-03

AI Summary

To address the lack of dedicated backdoor defense mechanisms for automatic speech recognition (ASR) models, this paper proposes a gradient-norm-based fine-tuning defense method. We first identify that backdoored neurons exhibit abnormally high gradients under trigger-injected inputs—a previously unobserved phenomenon in the audio domain—and leverage this insight to design the first domain-specific backdoor defense framework for ASR. Our approach suppresses anomalous neurons via gradient-norm regularization, augmented by efficient loss approximation and lightweight fine-tuning. Evaluated across five ASR models—including Wav2Vec 2.0 and DeepSpeech—and two speech datasets, the method achieves an average 92.7% reduction in backdoor attack success rate while incurring less than 0.5% degradation in clean-task accuracy. It consistently outperforms cross-domain transfer-based defenses in both robustness and utility.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Backdoor attacks have posed a significant threat to the security of deep neural networks (DNNs). Despite considerable strides in developing defenses against backdoor attacks in the visual domain, the specialized defenses for the audio domain remain empty. Furthermore, the defenses adapted from the visual to audio domain demonstrate limited effectiveness. To fill this gap, we propose Gradient Norm-based FineTuning (GN-FT), a novel defense strategy against the attacks in the audio domain, based on the observation from the corresponding backdoored models. Specifically, we first empirically find that the backdoored neurons exhibit greater gradient values compared to other neurons, while clean neurons stay the lowest. On this basis, we fine-tune the backdoored model by incorporating the gradient norm regularization, aiming to weaken and reduce the backdoored neurons. We further approximate the loss computation for lower implementation costs. Extensive experiments on two speech recognition datasets across five models demonstrate the superior performance of our proposed method. To the best of our knowledge, this work is the first specialized and effective defense against backdoor attacks in the audio domain.

Problem

• Backdoor Attacks
• Voice Data Processing
• Speech Recognition Security

Innovation

• Gradient Norm Fine-tuning
• Backdoor Defense
• Audio Data Security