Neural Additive Experts: Context-Gated Experts for Controllable Model Additivity

2026-02-11

AI Summary

This work addresses the limitation of generalized additive models (GAMs) in capturing feature interactions due to their strict additivity assumption, which often forces a trade-off between interpretability and predictive performance. To overcome this, the authors propose the Neural Additive Experts (NAE) framework, which constructs multiple expert networks for each input feature and employs a context-aware gating mechanism to dynamically integrate cross-feature information. By incorporating controllable additivity constraints and targeted regularization, NAE enables a smooth transition from purely additive representations to complex interaction modeling. Experimental results on both synthetic and real-world datasets demonstrate that NAE significantly improves prediction accuracy while preserving clear and reliable feature attributions, thereby effectively balancing model transparency with performance.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Multiagent Systems: Adversarial Agents
• Humans and AI: Explainable AI (XAI) for Human Understanding

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The trade-off between interpretability and accuracy remains a core challenge in machine learning. Standard Generalized Additive Models (GAMs) offer clear feature attributions but are often constrained by their strictly additive nature, which can limit predictive performance. Introducing feature interactions can boost accuracy yet may obscure individual feature contributions. To address these issues, we propose Neural Additive Experts (NAEs), a novel framework that seamlessly balances interpretability and accuracy. NAEs employ a mixture of experts framework, learning multiple specialized networks per feature, while a dynamic gating mechanism integrates information across features, thereby relaxing rigid additive constraints. Furthermore, we propose targeted regularization techniques to mitigate variance among expert predictions, facilitating a smooth transition from an exclusively additive model to one that captures intricate feature interactions while maintaining clarity in feature attributions. Our theoretical analysis and experiments on synthetic data illustrate the model's flexibility, and extensive evaluations on real-world datasets confirm that NAEs achieve an optimal balance between predictive accuracy and transparent, feature-level explanations. The code is available at https://github.com/Teddy-XiongGZ/NAE.

Problem

• interpretability
• accuracy
• Generalized Additive Models
• feature interactions
• model additivity

Innovation

• Neural Additive Experts
• Mixture of Experts
• Generalized Additive Models
• Feature Interactions
• Interpretability