AI chatbots versus human healthcare professionals: a systematic review and meta-analysis of empathy in patient care

2025-06-09

medRxiv

AI Summary

This study addresses the inconsistent and fragmented findings in current literature regarding the empathic performance of AI chatbots compared to human healthcare professionals. For the first time, it employs a systematic review and random-effects meta-analysis to quantitatively compare empathy in medical text-based interactions between large language model–based AI systems (e.g., ChatGPT-3.5/4) and humans. The analysis synthesizes data from 15 empirical studies published in 2023–2024, with risk of bias assessed using the ROBINS-I tool. Results demonstrate that AI significantly outperforms humans in empathy ratings (standardized mean difference = 0.87, 95% CI: 0.54–1.20, P < 0.00001), corresponding to an approximate two-point increase on a 10-point scale. This reveals a novel phenomenon wherein AI is perceived as more empathetic than human clinicians in specific healthcare communication contexts.

Technology Category

• Humans and AI: Emotional Intelligence
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Background: Empathy is widely recognised for improving patient outcomes ranging from reduced pain and anxiety to improved patient satisfaction, and its absence can cause harm. Meanwhile, use of artificial intelligence (AI)-based chatbots in healthcare is rapidly expanding, with one in five general practitioners (GPs) using generative AI to assist with tasks such as writing letters. Several studies suggest that these AI-based technologies are sometimes more empathic than human healthcare professionals (HCPs). However, the evidence in this area is mixed and has not been synthesised. Objective: To conduct a systematic review of studies that compare empathy of AI technologies with human HCP empathy. Methods: We searched multiple databases for studies comparing AI chatbots using large language models (e.g., GPT-3.5, GPT-4) with human HCPs on empathy measures. We assessed risk of bias with ROBINS-I and synthesised findings using random-effects meta-analysis where feasible, whilst avoiding double counting. Results: Our search identified 15 studies (2023-2024). Thirteen studies reported statistically significantly higher empathy ratings for AI, with only two studies situated in dermatology favouring human responses. Meta-analysis of 13 studies with data suitable for pooling, all utilising ChatGPT-3.5/4, showed a standardised mean difference (SMD) of 0.87 (95% CI, 0.54-1.20) favouring AI (p<0.00001). Conclusion: Our findings indicate that, in text-only scenarios, AI chatbots are frequently perceived as more empathic than human HCPs - equivalent to an increase of approximately two points on a 10-point empathy scale. Future research should validate these findings with direct patient evaluations and assess whether emerging voice-enabled AI systems can deliver similar empathic advantages.

Problem

• empathy
• AI chatbots
• healthcare professionals
• patient care
• systematic review

Innovation

• AI chatbots
• empathy
• meta-analysis
• large language models
• healthcare communication