The Most Important Features in Generalized Additive Models Might Be Groups of Features

2025-06-24

AI Summary

In interpretable machine learning, single-feature importance analysis often overlooks synergistic effects among feature groups—particularly problematic in multimodal or naturally grouped data, leading to missed critical collaborative signals. To address this, we propose the first post-hoc, retraining-free group importance measure for generalized additive models (GAMs), supporting user-defined and overlapping feature groups. Grounded in the statistical concept of explained variation, our metric ensures theoretical soundness while maintaining scalability to high-dimensional settings and preserving model interpretability. We validate the method on synthetic benchmarks and real-world multimodal neuroscience datasets—including depression classification and post-hip-replacement health monitoring—demonstrating substantial improvements in both comprehensiveness and accuracy of medical insights. Crucially, our approach uncovers feature-level collaboration patterns that remain invisible to conventional univariate importance analyses.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

While analyzing the importance of features has become ubiquitous in interpretable machine learning, the joint signal from a group of related features is sometimes overlooked or inadvertently excluded. Neglecting the joint signal could bypass a critical insight: in many instances, the most significant predictors are not isolated features, but rather the combined effect of groups of features. This can be especially problematic for datasets that contain natural groupings of features, including multimodal datasets. This paper introduces a novel approach to determine the importance of a group of features for Generalized Additive Models (GAMs) that is efficient, requires no model retraining, allows defining groups posthoc, permits overlapping groups, and remains meaningful in high-dimensional settings. Moreover, this definition offers a parallel with explained variation in statistics. We showcase properties of our method on three synthetic experiments that illustrate the behavior of group importance across various data regimes. We then demonstrate the importance of groups of features in identifying depressive symptoms from a multimodal neuroscience dataset, and study the importance of social determinants of health after total hip arthroplasty. These two case studies reveal that analyzing group importance offers a more accurate, holistic view of the medical issues compared to a single-feature analysis.

Problem

• Identifies joint signal from feature groups in GAMs
• Overcomes limitations of single-feature importance analysis
• Enhances interpretability for multimodal and high-dimensional datasets

Innovation

• Group feature importance in GAMs
• No retraining, posthoc overlapping groups
• Parallel with explained variation statistics