A Causal Lens for Evaluating Faithfulness Metrics

2025-02-26

AI Summary

Existing natural language explanations generated by large language models (LLMs) often fail to faithfully reflect their internal reasoning processes, and current faithfulness evaluation methods lack a unified, empirically verifiable framework. Method: We propose the first causality-based diagnostic framework for faithfulness evaluation: leveraging model editing techniques (ROME/MEMIT) to controllably generate paired faithful/unfaithful explanations, and systematically benchmarking mainstream explanation methods across four reasoning tasks—fact verification, analogy, counting, and multi-hop reasoning. Contribution/Results: This work pioneers the integration of causal intervention modeling into explanation evaluation, establishing a reproducible and verifiable faithfulness assessment paradigm. Empirical results show that all mainstream faithfulness metrics perform no better than random baselines. Our framework provides a critical diagnostic benchmark and actionable pathways for advancing trustworthy and interpretable AI.

Technology Category

• Reasoning under Uncertainty: Causality
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large Language Models (LLMs) offer natural language explanations as an alternative to feature attribution methods for model interpretability. However, despite their plausibility, they may not reflect the model's internal reasoning faithfully, which is crucial for understanding the model's true decision-making processes. Although several faithfulness metrics have been proposed, a unified evaluation framework remains absent. To address this gap, we present Causal Diagnosticity, a framework to evaluate faithfulness metrics for natural language explanations. Our framework employs the concept of causal diagnosticity, and uses model-editing methods to generate faithful-unfaithful explanation pairs. Our benchmark includes four tasks: fact-checking, analogy, object counting, and multi-hop reasoning. We evaluate a variety of faithfulness metrics, including post-hoc explanation and chain-of-thought-based methods. We find that all tested faithfulness metrics often fail to surpass a random baseline. Our work underscores the need for improved metrics and more reliable interpretability methods in LLMs.

Problem

• Evaluate faithfulness metrics for LLM explanations
• Propose Causal Diagnosticity framework
• Assess multiple interpretability methods in LLMs

Innovation

• Causal Diagnosticity framework
• model-editing methods
• faithfulness metrics evaluation