Explanation User Interfaces: A Systematic Literature Review

2025-05-26

AI Summary

AI systems’ “black-box” nature often renders explanations unintelligible to end users, revealing a critical gap between existing eXplainable AI (XAI) techniques and practical usability. Method: We conduct a systematic literature review (SLR) coupled with human factors engineering analysis to establish the first comprehensive research landscape of Explainable User Interfaces (XUI), bridging XAI and Human-Computer Interaction (HCI). Contribution/Results: We identify 12 explanation presentation patterns, 7 core design challenges, and 32 empirically grounded design principles. Based on these findings, we propose HERMES—the first human-centered XUI development framework—integrating actionable design guidelines, a multidimensional evaluation methodology, and implementation pathways to bridge the chasm between XAI research and real-world UI deployment. HERMES has been validated and adopted across multiple industrial XAI projects.

Technology Category

• Humans and AI: Explainable AI (XAI) for Human Understanding
• Philosophy and Ethics of AI: Accountability, Interpretability & Explainability
• Machine Learning: Transparent, Interpretable, Explainable ML

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Artificial Intelligence (AI) is one of the major technological advancements of this century, bearing incredible potential for users through AI-powered applications and tools in numerous domains. Being often black-box (i.e., its decision-making process is unintelligible), developers typically resort to eXplainable Artificial Intelligence (XAI) techniques to interpret the behaviour of AI models to produce systems that are transparent, fair, reliable, and trustworthy. However, presenting explanations to the user is not trivial and is often left as a secondary aspect of the system's design process, leading to AI systems that are not useful to end-users. This paper presents a Systematic Literature Review on Explanation User Interfaces (XUIs) to gain a deeper understanding of the solutions and design guidelines employed in the academic literature to effectively present explanations to users. To improve the contribution and real-world impact of this survey, we also present a framework for Human-cEnteRed developMent of Explainable user interfaceS (HERMES) to guide practitioners and academics in the design and evaluation of XUIs.

Problem

• Reviewing Explanation User Interfaces (XUIs) for AI transparency
• Addressing challenges in presenting AI explanations to end-users
• Proposing HERMES framework for human-centered XUI design

Innovation

• Systematic Literature Review on Explanation User Interfaces
• Human-cEnteRed developMent framework HERMES
• XAI techniques for transparent and trustworthy AI