Probing-Guided Layer Selection from Self-Supervised Speech Models for Generalizable Audio Deepfake Detection

2026-06-29

AI Summary

This work addresses the limited cross-domain generalization of existing audio deepfake detection models, which often rely on features tied to specific attacks or recording conditions. The authors propose a two-stage approach: first, a lightweight XGBoost probe evaluates and ranks the cross-domain discriminative capacity of individual layers in self-supervised speech models, revealing that informative representations are regionally clustered rather than confined to a single optimal layer; second, a compact classifier incorporating attention pooling and a shared bottleneck is constructed using selected layers, with the backbone frozen. This method is the first to identify generalizable deep regions within self-supervised models prior to training and enables a backbone-aware layer selection strategy. Using only four layers (1.34M trainable parameters) from XLS-R-300M, it achieves an EER of 4.94% on the In-The-Wild dataset and a cross-domain average EER of 5.07%, outperforming the current best frozen-backbone approach by 28%.

Abstract

Audio deepfake detection systems often fail to generalize across domains because they rely on features tied to specific attacks or recording conditions. Self-supervised speech models offer rich multi-layer representations, yet existing approaches either use a single layer or fuse all layers indiscriminately, and only reveal layer importance after training. We propose a model-agnostic, two-stage methodology that identifies informative depth zones before any task-specific model is trained. In the first stage, lightweight XGBoost probes evaluate each transformer layer's cross-domain discriminative power, producing a layer ranking. In the second stage, a compact neural classifier fuses only the selected layers through per-layer attention pooling and a shared bottleneck projection, while the backbone remains frozen. Applied across three backbones, the probing reveals two key findings. First, informative layers cluster in depth zones rather than at uniquely optimal positions: within-zone substitutions fall within multi-seed noise, while zone violations degrade performance by up to 5x. Second, the probing produces backbone-specific selections rather than a fixed layer recipe. On XLS-R-300M, four probing-selected layers with 1.34M trainable parameters achieve 4.94 +/- 0.32% equal error rate on In-The-Wild and 5.07% cross-domain average over four shared datasets, a 28% relative improvement over the best prior frozen-backbone result (Xiao and Vu, 2025) using all 25 layers with identical training data.

Problem

• audio deepfake detection
• cross-domain generalization
• self-supervised speech models
• layer selection
• representation learning

Innovation

• probing-guided layer selection
• self-supervised speech models
• audio deepfake detection
• cross-domain generalization
• attention pooling