Talk, Listen, Connect: Navigating Empathy in Human-AI Interactions

2024-09-23

arXiv.org

AI Summary

This study addresses the critical challenge of evaluating and enhancing AI empathic capability, systematically comparing empathic expression between humans and AI models (GPT-4o and fine-tuned variants) in mental health support contexts. Employing a mixed-methods approach, it integrates qualitative analysis of personal narratives, standardized empathic scoring, statistical modeling, and LLM supervised fine-tuning. We introduce the first multidimensional empathy assessment framework grounded in authentic social interaction requirements—filling a key gap in the quantitative evaluation of empathic authenticity in AI. Results reveal that current general-purpose AI exhibits weak contextual adaptation and biased emotional depth; targeted fine-tuning significantly improves empathic consistency and perceived credibility (p < 0.01). The work establishes a reproducible methodology and empirical foundation for designing and validating clinical-grade empathic AI systems for psychological support.

Technology Category

• Humans and AI: Emotional Intelligence
• Cognitive Modeling & Cognitive Systems: Affective Computing
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Social interactions promote well-being, yet challenges like geographic distance and mental health conditions can limit in-person engagement. Advances in AI agents are transferring communication, particularly in mental health, where AI chatbots provide accessible, non-judgmental support. However, a key challenge is how effectively these systems can express empathy, which is crucial in human-centered design. Current research highlights a gap in understanding how AI can authentically convey empathy, particularly as issues like anxiety, depression, and loneliness increase. Our research focuses on this gap by comparing empathy expression in human-human versus human-AI interactions. Using personal narratives and statistical analysis, we examine empathy levels elicited by humans and AI, including GPT-4o and fine-tuned versions of the model. This work aims to enhance the authenticity of AI-driven empathy, contributing to the future design of more reliable and effective mental health support systems that foster meaningful social interactions.

Problem

• Evaluating empathy perception in human-AI conversational interactions
• Assessing persona attributes and story qualities affecting empathy ratings
• Comparing AI-generated versus human responses to emotional narratives

Innovation

• Fine-tuned AI models for empathic response generation
• Evaluating persona attributes and story qualities on empathy
• Comparing AI and human sensitivity to emotional cues