From Questions to Insights: Exploring XAI Challenges Reported on Stack Overflow Questions

2025-04-03

AI Summary

Insufficient interpretability of AI models severely hinders their real-world deployment, while existing eXplainable AI (XAI) techniques—such as SHAP and LIME—face multiple usability challenges in practice. Method: We systematically construct the first taxonomy of XAI practical challenges based on 663 real-world Stack Overflow Q&A threads; propose a multidimensional severity assessment framework integrating answer quality and practitioner feedback; and validate findings via manual annotation, statistical testing, and developer interviews. Contribution/Results: We identify seven core bottlenecks, with “model integration difficulty” and “explanation disagreement” emerging as the most frequent and severe issues. We further distill “explanation consistency” and “simplified integration” as critical improvement directions, establishing the first developer-oriented XAI usability benchmark grounded in empirical practice.

Technology Category

• Humans and AI: Explainable AI (XAI) for Human Understanding
• Philosophy and Ethics of AI: Accountability, Interpretability & Explainability
• Machine Learning: Transparent, Interpretable, Explainable ML

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

The lack of interpretability is a major barrier that limits the practical usage of AI models. Several eXplainable AI (XAI) techniques (e.g., SHAP, LIME) have been employed to interpret these models' performance. However, users often face challenges when leveraging these techniques in real-world scenarios and thus submit questions in technical Q&A forums like Stack Overflow (SO) to resolve these challenges. We conducted an exploratory study to expose these challenges, their severity, and features that can make XAI techniques more accessible and easier to use. Our contributions to this study are fourfold. First, we manually analyzed 663 SO questions that discussed challenges related to XAI techniques. Our careful investigation produced a catalog of seven challenges (e.g., disagreement issues). We then analyzed their prevalence and found that model integration and disagreement issues emerged as the most prevalent challenges. Second, we attempt to estimate the severity of each XAI challenge by determining the correlation between challenge types and answer metadata (e.g., the presence of accepted answers). Our analysis suggests that model integration issues is the most severe challenge. Third, we attempt to perceive the severity of these challenges based on practitioners' ability to use XAI techniques effectively in their work. Practitioners' responses suggest that disagreement issues most severely affect the use of XAI techniques. Fourth, we seek agreement from practitioners on improvements or features that could make XAI techniques more accessible and user-friendly. The majority of them suggest consistency in explanations and simplified integration. Our study findings might (a) help to enhance the accessibility and usability of XAI and (b) act as the initial benchmark that can inspire future research.

Problem

• Identify challenges in using XAI techniques from Stack Overflow questions
• Assess severity of XAI challenges via answer metadata and practitioner feedback
• Propose improvements for XAI accessibility and usability based on practitioner input

Innovation

• Manual analysis of 663 Stack Overflow questions
• Correlation between challenge types and answer metadata
• Practitioners suggest consistency and simplified integration