Novel Loss-Enhanced Universal Adversarial Patches for Sustainable Speaker Privacy

2025-05-26

AI Summary

To address speaker identity leakage risks posed by deep learning speech models, this paper proposes a universal adversarial patch (UAP) method for voiceprint privacy protection. To overcome limitations of existing UAP approaches—including poor audio quality, severe automatic speech recognition (ASR) performance degradation, weak cross-model transferability, and strong dependence on input length—we introduce an exponential total variation (TV) loss function to enhance perturbation robustness and auditory imperceptibility. Additionally, we design a scalable UAP insertion mechanism, enabling stable defense against arbitrarily long speech inputs for the first time. Experiments demonstrate that our method achieves over 92% transfer success rates across multiple speaker verification models, while reducing PESQ degradation and ASR accuracy drop by more than 40% compared to prior methods—thereby significantly balancing privacy security and speech utility.

Technology Category

• Machine Learning: Privacy
• Natural Language Processing: Speech
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Security and Privacy: Large-scale security measurements
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Deep learning voice models are commonly used nowadays, but the safety processing of personal data, such as human identity and speech content, remains suspicious. To prevent malicious user identification, speaker anonymization methods were proposed. Current methods, particularly based on universal adversarial patch (UAP) applications, have drawbacks such as significant degradation of audio quality, decreased speech recognition quality, low transferability across different voice biometrics models, and performance dependence on the input audio length. To mitigate these drawbacks, in this work, we introduce and leverage the novel Exponential Total Variance (TV) loss function and provide experimental evidence that it positively affects UAP strength and imperceptibility. Moreover, we present a novel scalable UAP insertion procedure and demonstrate its uniformly high performance for various audio lengths.

Problem

• Prevent malicious identification in speaker anonymization methods
• Address audio quality degradation in universal adversarial patches
• Improve transferability across different voice biometrics models

Innovation

• Novel Exponential Total Variance loss function
• Scalable UAP insertion procedure
• Enhanced UAP strength and imperceptibility