Why Do You Say It Like That? A Phoneme-Level Framework for Explainable Speech Deepfake Detection

2026-07-09

AI Summary

This work addresses the limited interpretability of current deepfake speech detection models, which often fail to reveal the basis for their decisions. To enhance transparency, the authors propose a phoneme-level interpretable analysis framework that integrates Grad-CAM with automatic speech recognition, leveraging self-supervised speech representations from models such as wav2vec 2.0 or HuBERT to generate phoneme-aligned saliency maps. These maps effectively visualize acoustic anomalies in spoofed speech. Evaluated on the ASVspoof 5 dataset, the method achieves detection performance comparable to state-of-the-art models while uncovering statistically significant, attack-type- and speaker-dependent phoneme-level artifacts. This advancement substantially improves the interpretability and trustworthiness of deepfake detection systems by providing fine-grained, linguistically grounded explanations for model predictions.

Abstract

As the accuracy of speech deepfake detection improves with the use of self-supervised representations such as wav2vec 2.0 and HuBERT, understanding why the speech is classified as bona fide or deepfake remains an open challenge. In pursuit of more trustworthy and interpretable artificial intelligence, we introduce a phoneme-level analysis framework that connects model predictions to measurable phonetic units. Our post-hoc explainability method is generally applicable to a variety of speech deepfake detection systems based on convolutional neural networks since it leverages Gradient-weighted Class Activation Mapping in conjunction with speech recognition to generate saliency maps aligned with phonemes and pauses. This pipeline reveals statistically significant attack- and speaker-dependent phonetic cues associated with spoofed speech in terms that humans can understand. Experiments using ASVspoof 5 show comparable detection performance to similar architectures while providing linguistic interpretations across speakers and spoofing conditions.

Problem

• speech deepfake detection
• explainable AI
• phoneme-level analysis
• interpretability
• spoofing detection

Innovation

• phoneme-level explainability
• speech deepfake detection
• Grad-CAM
• self-supervised representations
• interpretable AI