Collab: Fostering Critical Identification of Deepfake Videos on Social Media via Synergistic Annotation

2026-01-24

AI Summary

This study addresses the dual challenges posed by the proliferation of deepfake videos on social media: the subtle and dynamic spatiotemporal artifacts they exhibit and the lack of effective identification tools for non-expert users. To this end, the authors propose a browser extension that enables collaborative annotation by lay users, integrating a novel confidence-weighted spatiotemporal IoU aggregation algorithm with a hierarchical attention-guided visualization mechanism. This work represents the first effort to combine collaborative annotation with attention guidance for deepfake detection. In a seven-day online experiment involving 90 participants, the proposed approach significantly improved both detection accuracy and users’ reflective awareness, outperforming baseline conditions lacking either aggregation or demonstrative guidance.

Technology Category

• Computer Vision: Video Understanding & Activity Analysis
• Application Domains: Misinformation & Fake News
• Humans and AI: Interaction Techniques and Devices

Application Category

• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity

Abstract

Identifying deepfake videos on social media platforms is challenged by dynamic spatio-temporal artifacts and inadequate user tools. This hinders both critical viewing by users and scalable moderation on platforms. Here, we present Collab, a web plugin enabling users to collaboratively annotate deepfake videos. Collab integrates three key components: (i) an intuitive interface for spatio-temporal labeling where users provide confidence scores and rationales, facilitating detailed input even from non-experts, (ii) a novel confidence-weighted spatio-temporal Intersection-over-Union (IoU) algorithm to aggregate diverse user annotations into accurate aggregations, and (iii) a hierarchical demonstration strategy presenting aggregated results to guide attention toward contentious regions and foster critical evaluation. A seven-day online study (N=90), where participants annotated suspicious videos when viewing an online experimental platforms, compared Collab against two conditions without aggregation or demonstration respectively. Collab significantly improved identification accuracy and enhanced reflection compared to non-demonstration condition, while outperforming non-aggregation condition for its novelty and effectiveness.

Problem

• deepfake
• social media
• video identification
• critical viewing
• content moderation

Innovation

• collaborative annotation
• deepfake detection
• spatio-temporal IoU
• confidence-weighted aggregation
• critical media literacy