Labeling Messages as AI-Generated Does Not Reduce Their Persuasive Effects

2025-04-14

AI Summary

This study examines whether AI-content labeling enhances public critical cognition and attenuates persuasive effects. Method: A randomized controlled experiment with 1,601 U.S. participants presented identical policy information, randomly labeled as “AI-generated,” “human expert-written,” or unlabeled. Results: Although 94.6% of participants trusted the label’s authenticity, labeling had no statistically significant effect on attitude change (+9.74 percentage points, p > 0.1), perceived accuracy, or sharing intention. These null effects were robust across political ideology, prior AI experience, and education level. This is the first high-statistical-power empirical study to challenge the transparency-intervention assumption that labeling inherently increases or decreases credibility. It demonstrates that merely disclosing AI authorship fails to activate expected cognitive regulatory mechanisms, providing critical boundary evidence for AI content governance and informing the design of more effective disclosure policies.

Technology Category

• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Application Domains: Humanities & Computational Social Science

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Responsible Web: Algorithmic accountability and transparency on the web
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content

Abstract

As generative artificial intelligence (AI) enables the creation and dissemination of information at massive scale and speed, it is increasingly important to understand how people perceive AI-generated content. One prominent policy proposal requires explicitly labeling AI-generated content to increase transparency and encourage critical thinking about the information, but prior research has not yet tested the effects of such labels. To address this gap, we conducted a survey experiment (N=1601) on a diverse sample of Americans, presenting participants with an AI-generated message about several public policies (e.g., allowing colleges to pay student-athletes), randomly assigning whether participants were told the message was generated by (a) an expert AI model, (b) a human policy expert, or (c) no label. We found that messages were generally persuasive, influencing participants' views of the policies by 9.74 percentage points on average. However, while 94.6% of participants assigned to the AI and human label conditions believed the authorship labels, labels had no significant effects on participants' attitude change toward the policies, judgments of message accuracy, nor intentions to share the message with others. These patterns were robust across a variety of participant characteristics, including prior knowledge of the policy, prior experience with AI, political party, education level, or age. Taken together, these results imply that, while authorship labels would likely enhance transparency, they are unlikely to substantially affect the persuasiveness of the labeled content, highlighting the need for alternative strategies to address challenges posed by AI-generated information.

Problem

• Testing if AI-generated content labels reduce persuasiveness
• Examining impact of authorship labels on message credibility
• Assessing effectiveness of transparency labels for AI information

Innovation

• Survey experiment tests AI label effects
• Random assignment of authorship labels
• Measures persuasiveness across diverse demographics