Teacher-Student Structure for Domain Adaptation in Ensemble Audio-Visual Video Deepfake Detection

2026-06-13

AI Summary

This study addresses the limited generalization of existing deepfake detection models in cross-domain scenarios. To this end, the authors propose EAV-DFD, a novel approach that introduces, for the first time, a teacher–student domain adaptation framework into multimodal deepfake detection. By integrating audio and visual features into an ensemble network, EAV-DFD significantly enhances detection performance on unseen target domains using only a small amount of labeled target-domain data. Beyond accurate forgery identification, the method also localizes the manipulated modality, thereby improving model interpretability. Experimental results demonstrate consistent improvements across three unseen datasets—DFDC, Deepfake_TIMIT, and PolyGlotFake—with AUC gains of 4.09%, 17.94%, and 0.5%, respectively.

Abstract

The rapid advancement of generative AI models is leading to more realistic deepfake media, encompassing the manipulation of audio, video, or both. This raises severe privacy and societal concerns. Numerous studies in this area have yielded promising intra-domain results; however, these models frequently exhibit decreased efficacy when faced with data from dissimilar domains. Consequently, recent deepfake detection approaches focus on enhancing the generalization ability through multiple techniques that incorporate all input modalities, including audio, images, and their interactions. In this regard, we propose the EAV-DFD method, a generalized deep ensemble audio-visual model (EAV-DFD) combined with a domain adaptation mechanism utilizing a teacher-student framework to enhance the model's ability to perform and generalize effectively across unseen domains. To evaluate the model's performance, we used the FakeAVCeleb dataset as the primary domain and the DFDC, Deepfake_TIMIT, and PolyGlotFake datasets as an unseen domain. Our experimental results demonstrate that the proposed framework is efficient in domain adaptation, improving AUC performance of the model by 4.09%, 17.94%, and 0.5% on three unseen datasets, using only a small portion of them to train the student model. This leads to a novel deepfake detection model capable of adapting to new domains and interpreting which modality has been manipulated, highlighting the potential of our approach for real-world applications.

Problem

• domain adaptation
• deepfake detection
• audio-visual
• generalization
• unseen domains

Innovation

• teacher-student framework
• domain adaptation
• audio-visual deepfake detection
• ensemble learning
• generalization