Effect of AI Performance, Risk Perception, and Trust on Human Dependence in Deepfake Detection AI system

2025-08-03

AI Summary

This study investigates how AI detection performance, user risk perception, and trust jointly influence human reliance on deepfake (image/audio/video) detection systems. A 400-participant online experiment integrated state-of-the-art AI detectors with behavioral tracking to quantify reliance dynamics under varying AI confidence levels and error conditions. Results show that AI accuracy significantly predicts both user trust and adoption rates; however, individual risk perception critically moderates this relationship—high-risk-perceivers calibrate AI recommendations more cautiously rather than complying uncritically. Based on these findings, we propose a “risk-perception-driven transparency design” framework for explainable AI, advocating the integration of uncertainty cues and risk metadata into system interfaces to support calibrated human-AI collaboration. The work provides empirical grounding and novel design principles for human-centered, trustworthy AI systems—advancing both human factors adaptation and interactive interface design in AI-assisted decision-making contexts.

Technology Category

• Humans and AI: Intelligent User Interfaces
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Computer Vision: Interpretability, Explainability, and Transparency

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Synthetic images, audio, and video can now be generated and edited by Artificial Intelligence (AI). In particular, the malicious use of synthetic data has raised concerns about potential harms to cybersecurity, personal privacy, and public trust. Although AI-based detection tools exist to help identify synthetic content, their limitations often lead to user mistrust and confusion between real and fake content. This study examines the role of AI performance in influencing human trust and decision making in synthetic data identification. Through an online human subject experiment involving 400 participants, we examined how varying AI performance impacts human trust and dependence on AI in deepfake detection. Our findings indicate how participants calibrate their dependence on AI based on their perceived risk and the prediction results provided by AI. These insights contribute to the development of transparent and explainable AI systems that better support everyday users in mitigating the harms of synthetic media.

Problem

• Examines AI performance impact on human trust in deepfake detection
• Studies how perceived risk influences human dependence on AI
• Aims to improve transparent AI systems for synthetic media identification

Innovation

• Examining AI performance impact on human trust
• Studying risk perception in deepfake detection
• Developing transparent explainable AI systems