Improving the Speaker Anonymization Evaluation's Robustness to Target Speakers with Adversarial Learning

2025-08-13

AI Summary

Existing privacy evaluation methods for voice anonymization—particularly those employing same-gender target speaker selection (TSA)—overestimate privacy protection by neglecting the coexistence of source- and target-speaker identity information in anonymized speech; moreover, TSA inherently leaks gender information, rendering anonymized outputs theoretically more vulnerable to re-identification attacks. Method: We propose a novel adversarial learning–based evaluation framework that introduces a target-speaker classifier jointly optimized with the source-speaker identifier to explicitly disentangle and suppress target identity cues, thereby enabling pure modeling of source-speaker identity. Contribution/Results: Our approach leverages deep neural networks to decompose speaker representations into source- and target-specific components. Extensive experiments across multiple anonymization systems demonstrate its effectiveness, especially in same-gender TSA scenarios, where it significantly improves both accuracy and robustness of privacy assessment. This work establishes a new paradigm for trustworthy voice anonymization evaluation.

Technology Category

• Machine Learning: Privacy
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Multiagent Systems: Adversarial Agents

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

The current privacy evaluation for speaker anonymization often overestimates privacy when a same-gender target selection algorithm (TSA) is used, although this TSA leaks the speaker's gender and should hence be more vulnerable. We hypothesize that this occurs because the evaluation does not account for the fact that anonymized speech contains information from both the source and target speakers. To address this, we propose to add a target classifier that measures the influence of target speaker information in the evaluation, which can also be removed with adversarial learning. Experiments demonstrate that this approach is effective for multiple anonymizers, particularly when using a same-gender TSA, leading to a more reliable assessment.

Problem

• Evaluating speaker anonymization overestimates privacy protection
• Same-gender target selection leaks speaker gender information
• Anonymized speech contains both source and target speaker information

Innovation

• Adversarial learning removes target speaker information
• Target classifier measures speaker influence in evaluation
• Same-gender target selection improves anonymization assessment