A Benchmark for Multi-speaker Anonymization

2024-07-08

arXiv.org

AI Summary

Existing voice anonymization methods primarily target single-speaker scenarios and exhibit insufficient privacy protection in multi-speaker settings—especially under speech overlap. This work presents the first systematic study of multi-speaker voice anonymization, establishing the first dedicated benchmark encompassing task formalization, evaluation protocols, baseline systems, and privacy leakage analysis. We propose a dialogue-level dual-objective vector anonymization framework that simultaneously preserves inter-speaker relational structure and enhances individual speaker distinguishability. Technically, it integrates spectral-clustering-based speaker diarization, disentangled anonymization, selective anonymizers, and two dialogue-level speaker vector optimization strategies. Experiments on both non-overlapping simulated and real-world datasets demonstrate substantial reductions in privacy leakage risk, alongside improvements in speech intelligibility and speaker discriminability.

Technology Category

• Machine Learning: Privacy
• Natural Language Processing: Speech
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Security and Privacy: Privacy-enhancing technologies
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Privacy-preserving voice protection approaches primarily suppress privacy-related information derived from paralinguistic attributes while preserving the linguistic content. Existing solutions focus particularly on single-speaker scenarios. However, they lack practicality for real-world applications, i.e., multi-speaker scenarios. In this paper, we present an initial attempt to provide a multi-speaker anonymization benchmark by defining the task and evaluation protocol, proposing benchmarking solutions, and discussing the privacy leakage of overlapping conversations. The proposed benchmark solutions are based on a cascaded system that integrates spectral-clustering-based speaker diarization and disentanglement-based speaker anonymization using a selection-based anonymizer. To improve utility, the benchmark solutions are further enhanced by two conversation-level speaker vector anonymization methods. The first method minimizes the differential similarity across speaker pairs in the original and anonymized conversations, which maintains original speaker relationships in the anonymized version. The other minimizes the aggregated similarity across anonymized speakers, which achieves better differentiation between speakers.Experiments conducted on both non-overlap simulated and real-world datasets demonstrate the effectiveness of the multi-speaker anonymization system with the proposed speaker anonymizers. Additionally, we analyzed overlapping speech regarding privacy leakage and provided potential solutions

Problem

• Develops benchmark for multi-speaker voice anonymization
• Addresses privacy leakage in overlapping conversations
• Enhances utility with speaker vector anonymization methods

Innovation

• Cascaded system with spectral-clustering speaker diarization
• Disentanglement-based speaker anonymization via selection
• Conversation-level speaker vector anonymization methods