What You Train Is What You Get: Gender Bias, Training Composition, and Post-Hoc Mitigation in Audio Deepfake Detection

2026-07-10

AI Summary

Although audio deepfake detection models achieve high overall accuracy, they exhibit significant gender bias. This study systematically investigates how the gender composition of training data influences such bias on the ASVspoof5 dataset and evaluates the effectiveness of six post-hoc calibration methods. It reveals for the first time that the gender ratio in the training set directly determines the direction of bias: WavLM features exacerbate the gender gap by 3–4.3 times compared to LogSpectrogram features. While balanced training mitigates bias for LogSpectrogram, it proves ineffective for WavLM. Critically, all calibration strategies—including the oracle approach—fail to eliminate a persistent 1.317% equal error rate disparity, demonstrating that fairness must be addressed during model training rather than as a post-processing step.

Abstract

Audio deepfake detection models determine whether speech is genuine or artificially generated, but high overall accuracy can mask substantial performance disparities across demographic groups. In this work, we investigate gender bias in audio deepfake detection using the ASVspoof5 dataset. We use ASVspoof5 under a controlled custom split designed to isolate gender-composition effects. We train attack-specific models on nine training sets with different gender compositions, ranging from female-only to male-only. We use a ResNet18 classifier with LogSpectrogram and WavLM-Base+ features, and we evaluated six post-hoc threshold calibration methods. Experimental results show that training data composition strongly predicts bias direction, with the underrepresented gender performing worse at test time. WavLM-Base+ features are shown to produce gender performance gaps 3.0 to 4.3 times larger than LogSpectrogram under identical training conditions, and balanced training is found to reduce LogSpectrogram bias but leave WavLM bias largely intact. Moreover, all six calibration strategies, including Oracle calibration with full test-set label access, leave the Equal Error Rate gap unchanged at 1.317 pp, confirming that threshold adjustment cannot correct underlying score distribution disparities. Overall, these findings suggest that gender fairness in audio deepfake detection must be addressed at training time, as post-hoc methods can only partially mitigate the resulting disparities

Problem

• gender bias
• audio deepfake detection
• training composition
• fairness
• demographic disparity

Innovation

• gender bias
• training composition
• audio deepfake detection
• post-hoc mitigation
• feature representation