Generative AI Literacy Training Improves Intelligence Analysts' Discrimination of Real and AI-Generated Images

2026-06-26

AI Summary

This study addresses the challenge intelligence analysts face in distinguishing authentic from AI-generated images, particularly their pronounced difficulty in correctly identifying real images. It presents the first evaluation of a 30-minute structured generative AI literacy intervention among professional intelligence personnel, which significantly enhanced their discrimination accuracy by highlighting diagnostic cues across seven real and fifty AI-generated images. Employing a within-subjects randomized controlled design with carefully matched image pairs varying in pose complexity and scene context, the research demonstrates a 9-percentage-point improvement in overall accuracy (from a 72% baseline), driven largely by a 14.2-percentage-point gain in correctly identifying real images. Based on 2,544 image-level judgments from 32 analysts, the findings reveal differential effects across experience levels and image types, underscoring the practical utility of brief, targeted training in professional settings.

Abstract

Across social and online platforms, people are increasingly exposed to AI-generated images. As a consequence, the task of distinguishing AI-generated from authentic images is becoming a central challenge for information ecosystems. While humans perform better than chance, accuracy falls short of many operational needs. Initial evidence shows that visually oriented training can improve deepfake detection but does not improve participants' ability to identify real images as real. Here, we investigate the efficacy of a brief training intervention for intelligence analysts employed by the United States government in 2024. We conducted a counterbalanced within-subject randomized experiment in which we showed participants real and AI-generated images varying in pose complexity and scene context and asked them whether each image was real or AI-generated, both before and after an expert delivered a 30-minute training that pointed out patterns in seven real and 50 AI-generated images. We collected 2,544 image-level judgments from 32 intelligence analysts. We find training increased overall accuracy by 9 percentage points (95% CI: [2.7, 15.4]) from a baseline of 72%. We find the improvement is driven by a 14.2 percentage point increase in accuracy for real images (95% CI: [0.7, 27.7]). Through a careful experimental setup that curated matched pairs of real and AI-generated images across pose complexity categories, we reveal how these trainings influence people with different levels of digital forensics and generative AI experience and identify the kind of image-based content where this training intervention appears to be most effective. Ultimately, these results provide causal evidence that a brief, structured training can improve human judgment across a diverse array of real and AI-generated images, informing organizational responses to AI-generated visual misinformation.

Problem

• AI-generated images
• image discrimination
• visual misinformation
• deepfake detection
• generative AI literacy

Innovation

• Generative AI Literacy
• Deepfake Detection
• Human-AI Discrimination
• Training Intervention
• Visual Misinformation