Not All Deepfakes Are Created Equal: Triaging Audio Forgeries for Robust Deepfake Singer Identification

2025-10-20

AI Summary

To address the growing risk of vocal identity infringement against artists posed by high-fidelity singing voice deepfakes, this paper proposes a two-stage singer identification framework. First, a forgery quality grading mechanism is established to filter out low-quality deepfake samples using a discriminative model; second, a lightweight speaker verification model is trained exclusively on authentic recordings to identify the original singer in highly realistic forgeries. By avoiding synthetic data—which introduces distributional shift—our approach significantly enhances robustness and attribution accuracy under deepfake conditions. Experiments demonstrate that the proposed system achieves substantially higher identification accuracy than state-of-the-art baselines on both genuine and high-fidelity synthetic singing audio. The framework thus offers strong practical utility and novelty in safeguarding artists’ vocal rights and ensuring audio content authenticity.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Computer Vision: Diffusion Models for Vision
• Machine Learning: Adversarial Learning & Robustness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity

Abstract

The proliferation of highly realistic singing voice deepfakes presents a significant challenge to protecting artist likeness and content authenticity. Automatic singer identification in vocal deepfakes is a promising avenue for artists and rights holders to defend against unauthorized use of their voice, but remains an open research problem. Based on the premise that the most harmful deepfakes are those of the highest quality, we introduce a two-stage pipeline to identify a singer's vocal likeness. It first employs a discriminator model to filter out low-quality forgeries that fail to accurately reproduce vocal likeness. A subsequent model, trained exclusively on authentic recordings, identifies the singer in the remaining high-quality deepfakes and authentic audio. Experiments show that this system consistently outperforms existing baselines on both authentic and synthetic content.

Problem

• Identifying singer vocal likeness in high-quality deepfakes
• Filtering low-quality forgeries to improve detection robustness
• Protecting artist voices from unauthorized synthetic content use

Innovation

• Two-stage pipeline for singer identification
• Discriminator model filters low-quality forgeries
• Singer model trained exclusively on authentic recordings