SocialDF: Benchmark Dataset and Detection Model for Mitigating Harmful Deepfake Content on Social Media Platforms

2025-06-05

AI Summary

Existing deepfake detection methods struggle to distinguish benign content from adversarial deepfakes deliberately designed to manipulate public perception—especially in realistic social media settings. This paper proposes a harm-oriented detection paradigm grounded in social context, introduces SocialDF—the first publicly available benchmark dataset covering multiple platforms and attack types—and designs a multimodal detection framework leveraging LLM-based multi-agent collaborative reasoning. The framework integrates face recognition, automatic speech recognition (ASR), cross-modal alignment, and contextual consistency analysis, enabling fine-grained harm assessment via multi-factor cross-validation. It achieves 92.7% accuracy on real-world social deepfake samples, significantly outperforming unimodal baselines. Key contributions are: (1) the first social-context-driven harm discrimination paradigm; (2) the open-sourced SocialDF benchmark—the first harm-oriented deepfake benchmark; and (3) empirical validation of LLM-based multi-agent collaborative reasoning for robust deepfake detection.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Multiagent Systems: Adversarial Agents

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

The rapid advancement of deep generative models has significantly improved the realism of synthetic media, presenting both opportunities and security challenges. While deepfake technology has valuable applications in entertainment and accessibility, it has emerged as a potent vector for misinformation campaigns, particularly on social media. Existing detection frameworks struggle to distinguish between benign and adversarially generated deepfakes engineered to manipulate public perception. To address this challenge, we introduce SocialDF, a curated dataset reflecting real-world deepfake challenges on social media platforms. This dataset encompasses high-fidelity deepfakes sourced from various online ecosystems, ensuring broad coverage of manipulative techniques. We propose a novel LLM-based multi-factor detection approach that combines facial recognition, automated speech transcription, and a multi-agent LLM pipeline to cross-verify audio-visual cues. Our methodology emphasizes robust, multi-modal verification techniques that incorporate linguistic, behavioral, and contextual analysis to effectively discern synthetic media from authentic content.

Problem

• Detect harmful deepfakes on social media platforms
• Distinguish benign from adversarial deepfake content
• Improve multi-modal verification of synthetic media

Innovation

• LLM-based multi-factor detection approach
• Multi-modal verification techniques
• Curated dataset SocialDF for real-world challenges