Cross-Branch Orthogonality for Improved Generalization in Face Deepfake Detection

2025-05-08

AI Summary

Existing deepfake detectors suffer from poor generalization and fail to reliably identify unseen forgery types. To address this, we propose a multi-branch feature learning framework that jointly models spatial cues—from coarse- to fine-grained—and semantic representations, while explicitly modeling their interactions for robust forgery discrimination. Our key contribution is a cross-branch orthogonality constraint, which decouples intra- and inter-branch features without introducing redundancy, thereby enhancing adaptability to unknown forgery methods. The framework is end-to-end trainable and integrates multi-scale spatial modeling with an orthogonality-aware loss. Extensive cross-dataset evaluations on Celeb-DF and DFDC demonstrate consistent improvements: +5% and +7% accuracy over state-of-the-art methods, respectively. These results validate substantial gains in both detection robustness and generalization capability.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Remarkable advancements in generative AI technology have given rise to a spectrum of novel deepfake categories with unprecedented leaps in their realism, and deepfakes are increasingly becoming a nuisance to law enforcement authorities and the general public. In particular, we observe alarming levels of confusion, deception, and loss of faith regarding multimedia content within society caused by face deepfakes, and existing deepfake detectors are struggling to keep up with the pace of improvements in deepfake generation. This is primarily due to their reliance on specific forgery artifacts, which limits their ability to generalise and detect novel deepfake types. To combat the spread of malicious face deepfakes, this paper proposes a new strategy that leverages coarse-to-fine spatial information, semantic information, and their interactions while ensuring feature distinctiveness and reducing the redundancy of the modelled features. A novel feature orthogonality-based disentanglement strategy is introduced to ensure branch-level and cross-branch feature disentanglement, which allows us to integrate multiple feature vectors without adding complexity to the feature space or compromising generalisation. Comprehensive experiments on three public benchmarks: FaceForensics++, Celeb-DF, and the Deepfake Detection Challenge (DFDC) show that these design choices enable the proposed approach to outperform current state-of-the-art methods by 5% on the Celeb-DF dataset and 7% on the DFDC dataset in a cross-dataset evaluation setting.

Problem

• Detecting novel deepfake types with improved generalization
• Reducing feature redundancy in face deepfake detection
• Enhancing cross-dataset performance for deepfake detectors

Innovation

• Leverages coarse-to-fine spatial and semantic information
• Introduces feature orthogonality-based disentanglement strategy
• Ensures branch-level and cross-branch feature distinctiveness