Meta Additive Model: Interpretable Sparse Learning With Auto Weighting

2026-04-21

AI Summary

Existing sparse additive models suffer significant performance degradation under complex perturbations such as non-Gaussian noise, outliers, label noise, and class imbalance, and often rely on manually designed sample reweighting strategies. This work proposes the Meta Additive Model (MAM), which for the first time integrates meta-learning with sparse additive modeling. Within a bilevel optimization framework, MAM employs a metadata-driven MLP to automatically learn sample-specific loss weights, enabling adaptive robust learning without pre-specified weighting functions. The method enjoys theoretical guarantees in terms of variable selection consistency, generalization ability, and algorithmic convergence. Extensive experiments on both synthetic and real-world datasets demonstrate that MAM substantially outperforms current state-of-the-art approaches, achieving superior predictive accuracy and robustness.

Technology Category

• Machine Learning: Active Learning
• Search and Optimization: Metareasoning and Metaheuristics
• Multiagent Systems: Adversarial Agents

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Sparse additive models have attracted much attention in high-dimensional data analysis due to their flexible representation and strong interpretability. However, most existing models are limited to single-level learning under the mean-squared error criterion, whose empirical performance can degrade significantly in the presence of complex noise, such as non-Gaussian perturbations, outliers, noisy labels, and imbalanced categories. The sample reweighting strategy is widely used to reduce the model's sensitivity to atypical data; however, it typically requires prespecifying the weighting functions and manually selecting additional hyperparameters. To address this issue, we propose a new meta additive model (MAM) based on the bilevel optimization framework, which learns data-driven weighting of individual losses by parameterizing the weighting function via an MLP trained on meta data. MAM is capable of a variety of learning tasks, including variable selection, robust regression estimation, and imbalanced classification. Theoretically, MAM provides guarantees on convergence in computation, algorithmic generalization, and variable selection consistency under mild conditions. Empirically, MAM outperforms several state-of-the-art additive models on both synthetic and real-world data under various data corruptions.

Problem

• sparse additive models
• complex noise
• sample reweighting
• hyperparameter selection
• non-Gaussian perturbations

Innovation

• meta additive model
• bilevel optimization
• auto weighting
• sparse learning
• robust regression