Generalizable Detection of Audio Deepfakes

2025-07-02

AI Summary

To address the limited generalization capability of audio deepfake detection models, this paper systematically investigates the synergistic optimization of pretrained backbone networks, data augmentation strategies, and loss functions. We propose lightweight, efficient detection architectures built upon Wav2Vec2, WavLM, and Whisper, integrated with joint time-domain and frequency-domain augmentations and a contrastive learning–driven loss design. Cross-scenario evaluations on ASVspoof2019–2021 and a private dataset demonstrate that our approach significantly improves robustness against unseen attack types and recording conditions. A single proposed model outperforms the winning system of the ASVspoof5 Challenge. This work establishes a reproducible, generalization-enhanced paradigm for audio deepfake detection and provides a standardized experimental benchmark framework.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

In this paper, we present our comprehensive study aimed at enhancing the generalization capabilities of audio deepfake detection models. We investigate the performance of various pre-trained backbones, including Wav2Vec2, WavLM, and Whisper, across a diverse set of datasets, including those from the ASVspoof challenges and additional sources. Our experiments focus on the effects of different data augmentation strategies and loss functions on model performance. The results of our research demonstrate substantial enhancements in the generalization capabilities of audio deepfake detection models, surpassing the performance of the top-ranked single system in the ASVspoof 5 Challenge. This study contributes valuable insights into the optimization of audio models for more robust deepfake detection and facilitates future research in this critical area.

Problem

• Enhancing generalization in audio deepfake detection models
• Evaluating pre-trained backbones across diverse datasets
• Optimizing data augmentation and loss functions

Innovation

• Uses pre-trained backbones like Wav2Vec2, WavLM, Whisper
• Explores data augmentation and loss functions
• Enhances generalization in audio deepfake detection