Who's Sorry Now: User Preferences Among Rote, Empathic, and Explanatory Apologies from LLM Chatbots

2025-07-03

AI Summary

This study investigates how large language model (LLM) chatbots can restore user trust after errors—specifically bias, factual inaccuracies, and hallucinations. Using a pre-registered within-subjects experiment (N = 162), we compared the efficacy of three apology strategies: mechanical, explanatory, and empathetic. Results show that explanatory apologies yield the highest overall trust restoration; empathetic apologies significantly outperform others in bias-related errors; and no significant preference emerges for hallucination errors—highlighting strong contextual dependence in trust repair. This work provides the first systematic empirical evidence of interaction effects between error type and apology strategy. We propose two design frameworks—“personalized apology” and “error–apology calibration”—to guide context-sensitive apology generation. These findings offer empirically grounded principles for developing trustworthy, human-centered AI dialogue systems.

Technology Category

• Natural Language Processing: Safety and Robustness
• Humans and AI: Learning Human Values and Preferences
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

As chatbots driven by large language models (LLMs) are increasingly deployed in everyday contexts, their ability to recover from errors through effective apologies is critical to maintaining user trust and satisfaction. In a preregistered study with Prolific workers (N=162), we examine user preferences for three types of apologies (rote, explanatory, and empathic) issued in response to three categories of common LLM mistakes (bias, unfounded fabrication, and factual errors). We designed a pairwise experiment in which participants evaluated chatbot responses consisting of an initial error, a subsequent apology, and a resolution. Explanatory apologies were generally preferred, but this varied by context and user. In the bias scenario, empathic apologies were favored for acknowledging emotional impact, while hallucinations, though seen as serious, elicited no clear preference, reflecting user uncertainty. Our findings show the complexity of effective apology in AI systems. We discuss key insights such as personalization and calibration that future systems must navigate to meaningfully repair trust.

Problem

• Examines user preferences for chatbot apology types
• Investigates effectiveness of apologies for LLM error categories
• Explores trust repair strategies in AI systems

Innovation

• Pairwise experiment design for user evaluation
• Explanatory apologies generally preferred by users
• Empathic apologies favored in bias scenarios