Fusion-SSAT: Unleashing the Potential of Self-supervised Auxiliary Task by Feature Fusion for Generalized Deepfake Detection

2026-01-02

arXiv.org

AI Summary

This work addresses the limited generalization capability of deepfake detection models when confronted with unseen forgery types and cross-dataset scenarios. To mitigate this issue, the authors propose a feature fusion approach that integrates self-supervised learning with multi-task learning. Specifically, a self-supervised task is incorporated as an auxiliary objective alongside the primary detection task during joint training, and an effective feature fusion strategy is designed to combine the representations from both tasks. This integration substantially enhances the model’s generalization performance. Extensive experiments demonstrate that the proposed method consistently outperforms state-of-the-art approaches across multiple benchmark datasets—including DF40, FaceForensics++, and Celeb-DF—with particularly notable gains in cross-dataset evaluation settings.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

In this work, we attempted to unleash the potential of self-supervised learning as an auxiliary task that can optimise the primary task of generalised deepfake detection. To explore this, we examined different combinations of the training schemes for these tasks that can be most effective. Our findings reveal that fusing the feature representation from self-supervised auxiliary tasks is a powerful feature representation for the problem at hand. Such a representation can leverage the ultimate potential and bring in a unique representation of both the self-supervised and primary tasks, achieving better performance for the primary task. We experimented on a large set of datasets, which includes DF40, FaceForensics++, Celeb-DF, DFD, FaceShifter, UADFV, and our results showed better generalizability on cross-dataset evaluation when compared with current state-of-the-art detectors.

Problem

• deepfake detection
• generalization
• self-supervised learning
• feature fusion
• cross-dataset evaluation

Innovation

• self-supervised learning
• feature fusion
• generalized deepfake detection
• auxiliary task
• cross-dataset generalization