CafGa: Customizing Feature Attributions to Explain Language Models

2025-09-25

AI Summary

Traditional feature attribution methods operate at a fixed word-level granularity, limiting their ability to efficiently capture cross-word semantic units and lacking user controllability. To address this, we propose an interactive, customizable-granularity attribution tool that enables users to define arbitrary text segments and visualize deletion/insertion attribution curves, facilitating dynamic and interpretable long-text analysis. Our approach is the first to integrate tunable granularity with established model-agnostic methods (e.g., SHAP, LIME), synergistically combining human-in-the-loop intervention with quantitative evaluation—thereby overcoming inherent limitations of token-level attribution. A user study demonstrates that our tool significantly improves explanation efficiency and readability, receiving broad endorsement across diverse user backgrounds and outperforming automated alternatives such as PartitionSHAP and MExGen.

Technology Category

• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Machine Learning: Transparent, Interpretable, Explainable ML
• Humans and AI: Explainable AI (XAI) for Human Understanding

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Feature attribution methods, such as SHAP and LIME, explain machine learning model predictions by quantifying the influence of each input component. When applying feature attributions to explain language models, a basic question is defining the interpretable components. Traditional feature attribution methods, commonly treat individual words as atomic units. This is highly computationally inefficient for long-form text and fails to capture semantic information that spans multiple words. To address this, we present CafGa, an interactive tool for generating and evaluating feature attribution explanations at customizable granularities. CafGa supports customized segmentation with user interaction and visualizes the deletion and insertion curves for explanation assessments. Through a user study involving participants of various expertise, we confirm CafGa's usefulness, particularly among LLM practitioners. Explanations created using CafGa were also perceived as more useful compared to those generated by two fully automatic baseline methods: PartitionSHAP and MExGen, suggesting the effectiveness of the system.

Problem

• Traditional feature attribution methods treat words as atomic units
• This approach is inefficient for long text and misses semantic spans
• CafGa enables customizable granularity explanations for language models

Innovation

• Customizable granularity for feature attribution explanations
• Interactive segmentation with user-defined text components
• Visual assessment via deletion and insertion curves