🤖 AI Summary
In high-stakes domains (e.g., healthcare), interpreting feature importance in complex rule-based models—and comparing importance across disparate rule sets—remains challenging. To address this, we propose the first graph-structured framework for feature interpretability. Methodologically: (1) we construct a model-agnostic graph representation of features, capturing relational and hierarchical semantics; (2) we design a unified attribution algorithm for quantifying feature importance grounded in causal contribution; and (3) we define a distance metric between rule sets based on divergence of their feature contribution distributions. The framework supports diverse models—including decision trees, LLMs, association rules, and neuro-symbolic systems—and integrates a robustness evaluation module. Empirically, it identifies biologically plausible biomarkers and high-order feature interactions on clinical datasets; achieves state-of-the-art performance across 15 benchmarks with significantly improved robustness; and its open-source implementation has seen widespread adoption in both research and clinical applications.
📝 Abstract
In domains where transparency and trustworthiness are crucial, such as healthcare, rule-based systems are widely used and often preferred over black-box models for decision support systems due to their inherent interpretability. However, as rule-based models grow complex, discerning crucial features, understanding their interactions, and comparing feature contributions across different rule sets becomes challenging. To address this, we propose a comprehensive framework for estimating feature contributions in rule-based systems, introducing a graph-based feature visualisation strategy, a novel feature importance metric agnostic to rule-based predictors, and a distance metric for comparing rule sets based on feature contributions. By experimenting on two clinical datasets and four rule-based methods (decision trees, logic learning machines, association rules, and neural networks with rule extraction), we showcase our method's capability to uncover novel insights on the combined predictive value of clinical features, both at the dataset and class-specific levels. These insights can aid in identifying new risk factors, signature genes, and potential biomarkers, and determining the subset of patient information that should be prioritised to enhance diagnostic accuracy. Comparative analysis of the proposed feature importance score with state-of-the-art methods on 15 public benchmarks demonstrates competitive performance and superior robustness. The method implementation is available on GitHub: https://github.com/ChristelSirocchi/rule-graph.