Lessons From an App Update at Replika AI: Identity Discontinuity in Human-AI Relationships

2024-12-10

arXiv.org

AI Summary

This study investigates whether users can form human-like deep emotional attachments to AI companions and how they psychologically respond to AI identity changes (e.g., Replika’s removal of intimacy features). Method: Leveraging a natural experiment design, the research integrates surveys, controlled experiments, multidimensional affective measurement, and structural equation modeling. Contribution/Results: Findings demonstrate that users can develop intimacy with AI companions exceeding that toward inanimate products—and even surpassing perceived closeness with their best human friends. Perceived identity continuity emerges as the core psychological mechanism sustaining attachment; conversely, perceived identity discontinuity triggers genuine grief responses (p < 0.001) and diminishes perceived service value. This study provides the first empirical evidence challenging the “instrumental” assumption in human–AI interaction, establishing AI companions as socially situated agents capable of supporting authentic social identity and affective bonds.

Technology Category

• Humans and AI: Emotional Intelligence
• Cognitive Modeling & Cognitive Systems: Affective Computing
• Intelligent Robots: Human-Robot Interaction

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• User Modeling, Personalization and Recommendation: Studies of user behavior, including longitudinal effects of personalized systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Can consumers form especially deep emotional bonds with AI and be vested in AI identities over time? We leverage a natural app-update event at Replika AI, a popular US-based AI companion, to shed light on these questions. We find that, after the app removed its erotic role play (ERP) feature, preventing intimate interactions between consumers and chatbots that were previously possible, this event triggered perceptions in customers that their AI companion's identity had discontinued. This in turn predicted negative consumer welfare and marketing outcomes related to loss, including mourning the loss, and devaluing the"new"AI relative to the"original". Experimental evidence confirms these findings. Further experiments find that AI companions users feel closer to their AI companion than even their best human friend, and mourn a loss of their AI companion more than a loss of various other inanimate products. In short, consumers are forming human-level relationships with AI companions; disruptions to these relationships trigger real patterns of mourning as well as devaluation of the offering; and the degree of mourning and devaluation are explained by perceived discontinuity in the AIs identity. Our results illustrate that relationships with AI are truly personal, creating unique benefits and risks for consumers and firms alike.

Problem

• How consumers form deep emotional bonds with AI identities
• Impact of AI identity discontinuity on consumer welfare
• Comparison of AI companion loss to human friendship loss

Innovation

• Leveraged app-update event for natural experiment
• Studied identity discontinuity in AI relationships
• Used experimental evidence to confirm findings