Automatic Generation of Explainability Requirements and Software Explanations From User Reviews

2025-07-09

AI Summary

Prior work lacks a systematic approach to transform unstructured explanation requests from user feedback into structured explainability requirements and corresponding explanations. Method: This paper proposes the first end-to-end automated framework—rule-guided prompt engineering—that integrates large language models (LLMs) and NLP techniques to jointly identify explanation needs, generate structured explainability requirements, and produce natural-language explanations from user reviews, augmented by human validation for quality assurance. Contribution/Results: We construct and publicly release the first annotated dataset of 58 real-world user comments. Experiments show that AI-generated explanations often surpass manually authored ones in clarity and stylistic expressiveness; however, requirement correctness remains error-prone and necessitates human correction—highlighting the critical role of human-AI collaboration. This work establishes a novel paradigm and empirical foundation for explainable AI (XAI) in requirements engineering.

Technology Category

• Humans and AI: Explainable AI (XAI) for Human Understanding
• Natural Language Processing: Prompt Engineering / Prompting
• Machine Learning: Transparent, Interpretable, Explainable ML

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Explainability has become a crucial non-functional requirement to enhance transparency, build user trust, and ensure regulatory compliance. However, translating explanation needs expressed in user feedback into structured requirements and corresponding explanations remains challenging. While existing methods can identify explanation-related concerns in user reviews, there is no established approach for systematically deriving requirements and generating aligned explanations. To contribute toward addressing this gap, we introduce a tool-supported approach that automates this process. To evaluate its effectiveness, we collaborated with an industrial automation manufacturer to create a dataset of 58 user reviews, each annotated with manually crafted explainability requirements and explanations. Our evaluation shows that while AI-generated requirements often lack relevance and correctness compared to human-created ones, the AI-generated explanations are frequently preferred for their clarity and style. Nonetheless, correctness remains an issue, highlighting the importance of human validation. This work contributes to the advancement of explainability requirements in software systems by (1) introducing an automated approach to derive requirements from user reviews and generate corresponding explanations, (2) providing empirical insights into the strengths and limitations of automatically generated artifacts, and (3) releasing a curated dataset to support future research on the automatic generation of explainability requirements.

Problem

• Automating explainability requirements from user reviews
• Generating clear software explanations automatically
• Addressing correctness gaps in AI-generated explanations

Innovation

• Automated explainability requirements from user reviews
• AI-generated explanations preferred for clarity
• Human validation ensures correctness and relevance