Phonetically Explainable Speech Deepfake Detection

2026-06-13

AI Summary

This work addresses the limitation of conventional speech spoofing detection methods that treat all temporal frames equally, thereby overlooking the varying discriminative power of different phonemes. To overcome this, the authors propose a phoneme-guided cross-attention framework that formulates detection as an inherently interpretable, phoneme-level decision process. Specifically, phoneme posteriorgrams serve as queries to selectively attend to relevant acoustic features via Transformer-based cross-attention, while phoneme prior weights enable structured aggregation of evidence. This approach uniquely embeds phoneme-level interpretability directly into the model architecture—eliminating the need for post-hoc explanations—and reveals that generative models are more prone to exposing artifacts in plosives and fricatives. The method achieves competitive performance on LJSpeech-derived datasets as well as ASVspoof 2019 LA and ASVspoof 5 Track 1, offering verifiable, phoneme-category-level decision rationales.

Abstract

Speech deepfake detection is predominantly treated as an opaque classification task where all temporal frames are aggregated equally. This ignores that different phonetic categories carry vastly different amounts of discriminative information. To address this, we propose a phoneme-guided cross-attention framework that transforms detection into an interpretable, phonetically grounded process. We factorize the spoofing posterior $P(\text{spoofed}\mid X, W)$, conditioned on the acoustic representation $X$ and the phonetic posteriorgram $W$. The resulting factorization can be written as $P(\text{spoofed} \mid X, W) = \sum_{i=1}^{M} w_i \cdot P(\text{spoofed} \mid X, Z = z_i)$, where $M$ denotes the number of phonetic classes, $P(\text{spoofed} \mid X, Z = z_i)$ is the spoofing probability for the $i$-th phonetic class $z_i$ conditioned on $X$, and each $w_i$ is the prevalence of phonetic class $z_i$ in the utterance. Our transformer-based architecture instantiates this through a cross-attention block in which phonetic queries selectively probe information in acoustic keys and values, with softmax-normalized pooling supplying explicit phone-presence weights. Unlike prior approaches that rely heavily on post-hoc explainability methods, our framework offers phonetic-explainability-by-design. We evaluate the framework on an LJSpeech-derived corpus, ASVspoof 2019 LA, and ASVspoof 5 Track 1. Per-phone importance rankings reveal that discriminative power concentrates on articulatory categories that generative models struggle to reproduce faithfully. Stops, fricatives, affricates, nasals, and silence-boundary closures rank most discriminative, while periodic vowels and semivowels rank lower. Beyond competitive performance, our model provides structural interpretability, yielding an inspectable per-articulatory category breakdown of the final verdict.

Problem

• speech deepfake detection
• phonetic explainability
• discriminative information
• temporal frame aggregation
• spoofing detection

Innovation

• phoneme-guided attention
• speech deepfake detection
• phonetic explainability
• cross-attention framework
• interpretable AI