Phoneme-Level Deepfake Detection Across Emotional Conditions Using Self-Supervised Embeddings

2026-05-04

AI Summary

This study addresses the limitations of existing audio deepfake detection methods, which often overlook the phonemic structure of speech and lack interpretability in emotional voice synthesis scenarios. To bridge this gap, the work introduces a phoneme-level fine-grained analysis framework that aligns genuine and synthetic utterances under matched emotional conditions. By integrating TextGrid-based phoneme alignment annotations with self-supervised embeddings from WavLM, the proposed approach enables effective detection of emotionally manipulated synthetic speech. Experimental results reveal that complex vowels and fricatives exhibit more pronounced deviations during synthesis, rendering them more discernible to detection models. The method demonstrates both strong effectiveness and enhanced interpretability across diverse emotional contexts and multiple text-to-speech systems.

Abstract

Recent advances in emotional voice conversion (EVC) have enabled the generation of expressive synthetic speech, raising new concerns in audio deepfake detection. Existing approaches treat speech as a homogeneous signal and largely overlook its internal phonetic structure, limiting their interpretability in emotionally conditioned settings. In this work, we propose a phoneme-level framework to analyze emotionally manipulated synthetic speech using real and EVC-generated speech under matched emotional conditions with shared transcripts, phoneme-aligned TextGrids, and WavLM-based embeddings. Our results show that phoneme behavior varies across categories, with complex vowels and fricatives exhibiting higher divergence while simpler phonemes remain more stable. Phonemes with larger distributional differences are also found to be more easily detected, consistently across multiple emotions and synthesis systems. These findings demonstrate that phoneme-level analysis is an effective and interpretable approach for detecting emotionally manipulated synthetic speech.

Problem

• deepfake detection
• emotional voice conversion
• phoneme-level analysis
• synthetic speech
• audio forensics

Innovation

• phoneme-level analysis
• emotional voice conversion
• self-supervised embeddings
• deepfake detection
• WavLM