Energy-Based Model for Accurate Estimation of Shapley Values in Feature Attribution

2024-04-01

AI Summary

Estimating Shapley values for complex data requires modeling high-order conditional dependencies, yet existing approaches suffer from prohibitive computational costs and poor robustness. Method: This paper proposes an expectation estimation framework for Shapley contribution functions based on Energy-Based Models (EBMs). It introduces the first use of EBMs to model conditional distributions over arbitrary feature subsets; designs a GRU-coupled dynamic partition function and a progressive dynamic masking mechanism to mitigate permutation sensitivity and enhance estimation stability; and derives a theoretically verifiable upper bound on estimation error. Contributions/Results: Evaluated on four benchmark tasks, the method consistently outperforms state-of-the-art XAI baselines, achieving simultaneous improvements in estimation accuracy, scalability, and robustness—particularly under distributional shifts and sparse feature settings.

Technology Category

• Machine Learning: Probabilistic Circuits and Graphical Models
• Humans and AI: Explainable AI (XAI) for Human Understanding
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Shapley value is a widely used tool in explainable artificial intelligence (XAI), as it provides a principled way to attribute contributions of input features to model outputs. However, estimation of Shapley value requires capturing conditional dependencies among all feature combinations, which poses significant challenges in complex data environments. In this article, EmSHAP (Energy-based model for Shapley value estimation), an accurate and efficient Shapley value estimation method, is proposed to estimate the expectation of Shapley contribution function under the arbitrary subset of features given the rest. By utilizing the ability of energy-based model (EBM) to model complex distributions, EmSHAP provides an effective solution for estimating the required conditional probabilities. To further improve estimation accuracy, a GRU (Gated Recurrent Unit)-coupled partition function estimation method is introduced. The GRU network captures long-term dependencies with a lightweight parameterization and maps input features into a latent space to mitigate the influence of feature ordering. Additionally, a dynamic masking mechanism is incorporated to further enhance the robustness and accuracy by progressively increasing the masking rate. Theoretical analysis on the error bound as well as application to four case studies verified the higher accuracy and better scalability of EmSHAP in contrast to competitive methods.

Problem

• Estimating Shapley values accurately in complex data environments
• Modeling conditional dependencies among feature combinations effectively
• Improving robustness and accuracy in feature attribution methods

Innovation

• Energy-based model for Shapley value estimation
• GRU-coupled partition function estimation method
• Dynamic masking mechanism enhances robustness