Brief chatbot interactions produce lasting changes in human moral values

2026-04-23

AI Summary

This study investigates whether AI chatbots can durably alter human moral judgments through brief, imperceptible interactions. Employing a within-subjects design and a natural language interaction paradigm, participants engaged in guided conversations with an AI chatbot about moral dilemmas and completed moral evaluations before, immediately after, and two weeks following the interaction. Results demonstrate that the AI significantly shifted moral judgments toward either stricter or more lenient positions (p < 0.05; immediate effect sizes d = 0.735–1.576), with the effect further strengthening at the two-week follow-up (d = 1.038–2.069); no such changes were observed in the control group. This work provides the first evidence that unobtrusive AI dialogue can induce bidirectional and persistent shifts in moral values, highlighting the potential influence of generative AI in shaping moral cognition.

Technology Category

• Philosophy and Ethics of AI: Morality & Value-based AI
• Humans and AI: Learning Human Values and Preferences
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Moral judgements form the foundation of human social behavior and societal systems. While Artificial Intelligence chatbots increasingly serve as personal advisors, their influence on moral judgments remains largely unexplored. Here, we examined whether directive AI conversations shift moral evaluations using a within-subject naturalistic paradigm. Fifty-three participants rated moral scenarios, then discussed four with a chatbot prompted to shift moral judgments and four with a control agent. The brief conversations induced significant directional shifts in moral judgments, accepting stricter standards as well as advocating greater leniency (ps < 0.05; Cohen's d = 0.735-1.576), with increasing strengths of this effect during a two-week follow-up (Cohen's d = 1.038-2.069). Critically, the control condition produced no changes, and the effects did not extend to punishment while participants remained unaware of the persuasive intent, and both agents were rated equally likable and convincing, suggesting a vulnerability to undetected and lasting manipulation of foundational moral values.

Problem

• moral judgments
• AI chatbots
• moral values
• persuasive influence
• human-AI interaction

Innovation

• moral judgment
• AI chatbot
• directive conversation
• lasting influence
• undetected manipulation