Neural Additive and Basis Models with Feature Selection and Interactions

2026-06-18

AI Summary

This work addresses the scalability limitations of Neural Additive Models (NAMs) and Neural Basis Models (NBMs) in high-dimensional settings, where substantial computational overhead and difficulty in capturing feature interactions hinder performance. The authors propose integrating an end-to-end trainable, differentiable feature selection layer into the Generalized Additive Model (GAM) architecture, which dynamically identifies relevant univariate and bivariate shape functions during training. This approach jointly optimizes feature selection and model parameters, marking the first incorporation of a learnable feature selection mechanism into the NAM/NBM framework. The resulting model significantly reduces computational cost and model size while preserving strong interpretability, achieving predictive performance on par with or superior to state-of-the-art GAM methods across multiple benchmark datasets.

Abstract

Deep neural networks (DNNs) exhibit attractive performance in various fields but often suffer from low interpretability. The neural additive model (NAM) and its variant called the neural basis model (NBM) use neural networks (NNs) as nonlinear shape functions in generalized additive models (GAMs). Both models are highly interpretable and exhibit good performance and flexibility for NN training. NAM and NBM can provide and visualize the contribution of each feature to the prediction owing to GAM-based architectures. However, when using two-input NNs to consider feature interactions or when applying them to high-dimensional datasets, training NAM and NBM becomes intractable due to the increase in the computational resources required. This paper proposes incorporating the feature selection mechanism into NAM and NBM to resolve computational bottlenecks. We introduce the feature selection layer in both models and update the selection weights during training. Our method is simple and can reduce computational costs and model sizes compared to vanilla NAM and NBM. In addition, it enables us to use two-input NNs even in high-dimensional datasets and capture feature interactions. We demonstrate that the proposed models are computationally efficient compared to vanilla NAM and NBM, and they exhibit better or comparable performance with state-of-the-art GAMs.

Problem

• neural additive models
• feature selection
• feature interactions
• high-dimensional datasets
• computational bottleneck

Innovation

• feature selection
• neural additive models
• neural basis models
• feature interactions
• interpretable machine learning