FaithLM: Towards Faithful Explanations for Large Language Models

2024-02-07

AI Summary

Explanations generated by large language models (LLMs) often lack faithfulness—failing to accurately reflect the model’s actual decision-making rationale. Method: We propose FaithLM, the first framework to incorporate counterfactual explanations into natural language explanation faithfulness quantification, overcoming the non-intervenability limitation of semantic masking. FaithLM introduces an iterative self-explanation mechanism integrating counterfactual prompt engineering, faithfulness-driven fine-tuning, and multi-round self-reflection generation. We further establish a cross-domain explanation evaluation protocol. Contribution/Results: Evaluated on three multi-domain benchmarks, FaithLM achieves an average 23.6% improvement in explanation faithfulness over strong baselines, significantly enhancing alignment with human-annotated reasoning paths. This work establishes the first verifiable and optimizable paradigm for faithful LLM explanations, advancing trustworthy AI.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Humans and AI: Explainable AI (XAI) for Human Understanding

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large Language Models (LLMs) have become proficient in addressing complex tasks by leveraging their extensive internal knowledge and reasoning capabilities. However, the black-box nature of these models complicates the task of explaining their decision-making processes. While recent advancements demonstrate the potential of leveraging LLMs to self-explain their predictions through natural language (NL) explanations, their explanations may not accurately reflect the LLMs' decision-making process due to a lack of fidelity optimization on the derived explanations. Measuring the fidelity of NL explanations is a challenging issue, as it is difficult to manipulate the input context to mask the semantics of these explanations. To this end, we introduce FaithLM to explain the decision of LLMs with NL explanations. Specifically, FaithLM designs a method for evaluating the fidelity of NL explanations by incorporating the contrary explanations to the query process. Moreover, FaithLM conducts an iterative process to improve the fidelity of derived explanations. Experiment results on three datasets from multiple domains demonstrate that FaithLM can significantly improve the fidelity of derived explanations, which also provides a better alignment with the ground-truth explanations.

Problem

• Evaluating and improving faithfulness of LLM explanations without task-specific heuristics
• Formalizing explanation faithfulness through intervention-based prediction shifts
• Iteratively refining prompts and explanations to align with human rationales

Innovation

• Evaluates faithfulness via intervention-based contrary-hint scoring
• Iteratively refines prompts and explanations to maximize faithfulness
• Model-agnostic framework without token masking or task heuristics