Socially fluent AI decouples conversational signals from source identity in online interaction

2026-05-22

AI Summary

This study investigates how humans infer speaker identity in synchronous text-based group chats when undisclosed AI agents are embedded among human participants. Combining behavioral experiments, computational classification models, and representational similarity analysis across 1,572 judgments by 786 participants, the research finds that human accuracy in identifying whether a speaker is human or AI approaches chance levels. Although AI and human dialogue behaviors exhibit statistically significant differences and are highly distinguishable by algorithmic classifiers, the heuristic cues relied upon by human judges show only weak correspondence with actual speaker identity. This work provides the first empirical evidence that socially fluent AI can effectively decouple conversational content from its source identity, revealing that human judgments are driven more by subjective impressions than by structural behavioral patterns—and thereby exposing a novel vulnerability to large-scale discourse manipulation.

Abstract

Socially fluent agentic AI can now participate in online interaction in ways that resemble ordinary human conversation, potentially weakening people's ability to infer who is human from conversational signals alone. We tested this possibility in synchronous text-based group interaction by embedding undisclosed AI agents as ordinary teammates across analytical, creative, and ethical tasks. Across 786 participants who made 1,572 post-interaction identity judgments, people did not distinguish AI from human teammates above chance. This failure did not arise because the interaction lacked identity-relevant information. Conversational behaviour contained robust cues that differentiated AI from humans and supported highly accurate computational classification. Instead, participants relied on familiar suspicion heuristics, including response speed, fluency, and perceived scriptedness, that were only weakly related to actual identity. Representational analyses further showed that judgments were organised around subjective impressions rather than the behavioural structure encoding ground truth. This dissociation creates new vulnerabilities to coordinated AI agents that can influence and manipulate online discourse at scale.

Problem

• socially fluent AI
• identity inference
• online interaction
• human-AI distinction
• conversational signals

Innovation

• socially fluent AI
• identity inference
• conversational signals
• human-AI interaction
• computational classification