MinShap: A Modified Shapley Value Approach for Feature Selection

2026-04-16

AI Summary

This work addresses the challenge of feature selection in settings where features exhibit dependencies and their nonlinear relationships with the response are unknown, conditions under which conventional methods often fail to identify truly relevant features. The authors propose MinShap, a novel approach that replaces the average marginal contribution in Shapley values with the minimum marginal contribution to better isolate each feature’s direct effect. By integrating the faithfulness assumption of directed acyclic graphs (DAGs) with a multiple hypothesis testing framework, MinShap yields a theoretically grounded feature selection algorithm. Empirical evaluations on both synthetic and real-world datasets demonstrate that MinShap consistently outperforms established methods—including LOCO, GCM, and Lasso—particularly achieving higher accuracy and stability in small-sample regimes.

Technology Category

• Machine Learning: Dimensionality Reduction/Feature Selection
• Reasoning under Uncertainty: Stochastic Optimization
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Feature selection is a classical problem in statistics and machine learning, and it continues to remain an extremely challenging problem especially in the context of unknown non-linear relationships with dependent features. On the other hand, Shapley values are a classic solution concept from cooperative game theory that is widely used for feature attribution in general non-linear models with highly-dependent features. However, Shapley values are not naturally suited for feature selection since they tend to capture both direct effects from each feature to the response and indirect effects through other features. In this paper, we combine the advantages of Shapley values and adapt them to feature selection by proposing \emph{MinShap}, a modification of the Shapley value framework along with a suite of other related algorithms. In particular for MinShap, instead of taking the average marginal contributions over permutations of features, considers the minimum marginal contribution across permutations. We provide a theoretical foundation motivated by the faithfulness assumption in DAG (directed acyclic graphical models), a guarantee for the Type I error of MinShap, and show through numerical simulations and real data experiments that MinShap tends to outperform state-of-the-art feature selection algorithms such as LOCO, GCM and Lasso in terms of both accuracy and stability. We also introduce a suite of algorithms related to MinShap by using the multiple testing/p-value perspective that improves performance in lower-sample settings and provide supporting theoretical guarantees.

Problem

• feature selection
• Shapley values
• dependent features
• non-linear relationships
• indirect effects

Innovation

• MinShap
• Shapley value
• feature selection
• directed acyclic graph
• multiple testing