Response and Prompt Evaluation to Prevent Parasocial Relationships with Chatbots

2025-08-21

AI Summary

Parasocial relationships with AI agents may adversely affect users’ mental health, necessitating early detection and intervention during human-AI interactions. To address this, we propose a lightweight, large language model (LLM)-based real-time response assessment framework that jointly models user prompts and system responses to dynamically detect parasocial cues. Our approach introduces a synthetically constructed dialogue dataset and employs an iterative five-stage testing methodology to rigorously validate effectiveness. Experiments across 30 realistic simulated dialogues achieve a 100% true positive rate: all high-risk instances are accurately identified within the first three interaction turns, with zero false positives. The framework demonstrates both high sensitivity and strong robustness—achieving reliable detection without compromising responsiveness or scalability. This work provides a deployable technical pathway for designing safe, controllable social AI systems.

Technology Category

• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction
• Humans and AI: Interaction Techniques and Devices
• Multiagent Systems: Adversarial Agents

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

The development of parasocial relationships with AI agents has severe, and in some cases, tragic effects for human well-being. Yet preventing such dynamics is challenging: parasocial cues often emerge gradually in private conversations, and not all forms of emotional engagement are inherently harmful. We address this challenge by introducing a simple response evaluation framework, created by repurposing a state-of-the-art language model, that evaluates ongoing conversations for parasocial cues in real time. To test the feasibility of this approach, we constructed a small synthetic dataset of thirty dialogues spanning parasocial, sycophantic, and neutral conversations. Iterative evaluation with five stage testing successfully identified all parasocial conversations while avoiding false positives under a tolerant unanimity rule, with detection typically occurring within the first few exchanges. These findings provide preliminary evidence that evaluation agents can provide a viable solution for the prevention of parasocial relations.

Problem

• Preventing harmful parasocial relationships with AI chatbots
• Detecting parasocial cues in real-time conversations
• Evaluating emotional engagement to avoid false positives

Innovation

• Real-time parasocial cue evaluation framework
• Repurposed state-of-the-art language model
• Five stage testing with synthetic dialogues