Forensic Similarity for Speech Deepfakes

2025-10-03

AI Summary

This work addresses the forensic provenance problem of speech deepfakes by proposing the first forensic similarity analysis method for deepfake audio, designed to determine whether two audio samples originate from the same generative model. The method employs a two-stage deep network: first extracting robust forensic features using a pretrained deepfake detector, then computing a consistency score via a lightweight similarity network. Crucially, it requires no assumptions about or training on specific forgery artifacts, enabling strong generalization to unseen deepfake techniques. Evaluated on source verification, it significantly outperforms baseline methods and supports extended applications such as splice detection. Its core contribution lies in pioneering the forensic similarity paradigm for speech deepfakes—overcoming traditional limitations that rely on known artifacts or model priors—while ensuring robustness, adaptability, and practical utility.

Technology Category

• Natural Language Processing: Speech
• Machine Learning: Deep Generative Models & Autoencoders
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

In this paper, we introduce a digital audio forensics approach called Forensic Similarity for Speech Deepfakes, which determines whether two audio segments contain the same forensic traces or not. Our work is inspired by prior work in the image domain on forensic similarity, which proved strong generalization capabilities against unknown forensic traces, without requiring prior knowledge of them at training time. To achieve this in the audio setting, we propose a two-part deep-learning system composed of a feature extractor based on a speech deepfake detector backbone and a shallow neural network, referred to as the similarity network. This system maps pairs of audio segments to a score indicating whether they contain the same or different forensic traces. We evaluate the system on the emerging task of source verification, highlighting its ability to identify whether two samples originate from the same generative model. Additionally, we assess its applicability to splicing detection as a complementary use case. Experiments show that the method generalizes to a wide range of forensic traces, including previously unseen ones, illustrating its flexibility and practical value in digital audio forensics.

Problem

• Detecting shared forensic traces in audio segments
• Verifying if speech samples originate from same generative model
• Identifying audio manipulations through deepfake detection system

Innovation

• Deep learning system with feature extractor and similarity network
• Maps audio pairs to forensic trace similarity scores
• Generalizes to unseen forensic traces without prior knowledge