Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Era

2024-03-13

arXiv.org

AI Summary

Despite growing interest in eXplainable Artificial Intelligence (XAI), its practical deployment remains challenging in the large language model (LLM) era due to misalignment between explanations and user needs, lack of interactivity, and insufficient integration with LLM capabilities. Method: This paper proposes the “Usable XAI” paradigm—a bidirectional LLM–XAI co-enhancement framework. It leverages XAI to improve LLM trustworthiness and utility while harnessing LLMs to revolutionize explanation generation, interactive interpretation, and human-centered evaluation. Technically, it integrates prompt engineering, chain-of-thought reasoning, explainability-aware fine-tuning, human-in-the-loop evaluation, and LLM-driven explanation rewriting and visualization into an end-to-end explainability enhancement pipeline. Contribution/Results: We introduce the first systematic set of ten co-evolution strategies for LLM–XAI integration, enabling automated, intent-aligned, and dynamically optimized explanations. Empirical validation across multiple real-world scenarios demonstrates significant improvements in task completion rates and user trust. We publicly release code and practical guidelines to bridge the gap between XAI theory and engineering practice.

Technology Category

• Humans and AI: Explainable AI (XAI) for Human Understanding
• Machine Learning: Large Multimodal Models (LMMs)
• Philosophy and Ethics of AI: Accountability, Interpretability & Explainability

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Explainable AI (XAI) refers to techniques that provide human-understandable insights into the workings of AI models. Recently, the focus of XAI is being extended toward explaining Large Language Models (LLMs). This extension calls for a significant transformation in the XAI methodologies for two reasons. First, many existing XAI methods cannot be directly applied to LLMs due to their complexity and advanced capabilities. Second, as LLMs are increasingly deployed in diverse applications, the role of XAI shifts from merely opening the ``black box'' to actively enhancing the productivity and applicability of LLMs in real-world settings. Meanwhile, the conversation and generation abilities of LLMs can reciprocally enhance XAI. Therefore, in this paper, we introduce Usable XAI in the context of LLMs by analyzing (1) how XAI can explain and improve LLM-based AI systems and (2) how XAI techniques can be improved by using LLMs. We introduce 10 strategies, introducing the key techniques for each and discussing their associated challenges. We also provide case studies to demonstrate how to obtain and leverage explanations. The code used in this paper can be found at: https://github.com/JacksonWuxs/UsableXAI_LLM.

Problem

• Extending XAI methods for complex Large Language Models (LLMs)
• Shifting XAI role from transparency to enhancing LLM productivity
• Reciprocal improvement between LLMs and XAI techniques

Innovation

• Extends XAI to explain complex Large Language Models
• Introduces 10 strategies for usable XAI in LLMs
• Leverages LLMs to enhance XAI techniques