AWARE-NET: adaptive weighted averaging for robust ensemble network in deepfake detection

2025-04-01

IET Conference Proceedings

AI Summary

To address weak cross-dataset and cross-manipulation-type generalization in deepfake detection, this paper proposes a two-level adaptive weighted ensemble framework. At the first level, three randomly initialized models of the same architecture (Xception, Res2Net101, EfficientNet-B7) are ensembled via mean fusion to reduce variance. At the second level, learnable backpropagation-based weighting dynamically assigns architecture-level weights according to each model’s reliability, enabling adaptive fusion. This work is the first to jointly integrate hierarchical weighting with diversity enhancement via random initialization. The method achieves state-of-the-art performance on FF++ and CelebDF-v2 (AUC = 100.00%, F1 = 99.95%). For cross-dataset generalization, it attains AUCs of 88.20% and 72.52%—substantially outperforming existing approaches.

Technology Category

• Machine Learning: Ensemble Methods
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Deepfake detection has become increasingly important due to the rise of synthetic media, which poses significant risks to digital identity and cyber presence for security and trust. While multiple approaches have improved detection accuracy, challenges remain in achieving consistent performance across diverse datasets and manipulation types. In response, we propose a novel two-tier ensemble framework for deepfake detection based on deep learning that hierarchically combines multiple instances of three state-of-the-art architectures: Xception, Res2Net101, and EfficientNet-B7. Our framework employs a unique approach where each architecture is instantiated three times with different initializations to enhance model diversity, followed by a learnable weighting mechanism that dynamically combines their predictions. Unlike traditional fixed-weight ensembles, our first-tier averages predictions within each architecture family to reduce model variance, while the second tier learns optimal contribution weights through backpropagation, automatically adjusting each architecture's influence based on their detection reliability. Our experiments achieved state-of-the-art intra-dataset performance with AUC scores of 99.22% (FF++) and 100.00% (CelebDF-v2), and F1 scores of 98.06% (FF++) and 99.94% (CelebDF-v2) without augmentation. With augmentation, we achieve AUC scores of 99.47% (FF++) and 100.00% (CelebDF-v2), and F1 scores of 98.43% (FF++) and 99.95% (CelebDF-v2). The framework demonstrates robust cross-dataset generalization, achieving AUC scores of 88.20% and 72.52%, and F1 scores of 93.16% and 80.62% in cross-dataset evaluations.

Problem

• Improving deepfake detection accuracy across diverse datasets
• Enhancing consistency in detecting various manipulation types
• Developing adaptive ensemble framework for robust generalization

Innovation

• Two-tier ensemble framework combining Xception, Res2Net101, EfficientNet-B7
• Learnable weighting mechanism dynamically adjusts architecture contributions
• Multiple initializations enhance model diversity for robust detection