S2MAM: Semi-supervised Meta Additive Model for Robust Estimation and Variable Selection

2026-04-21

AI Summary

This work addresses the sensitivity of graph Laplacian regularization in semi-supervised learning to predefined similarity measures and its vulnerability to redundant or noisy variables, which compromises model robustness and interpretability. The authors propose a novel bi-level optimization framework for a semi-supervised meta additive model that jointly performs variable selection and similarity graph learning. In this framework, the outer loop automatically identifies informative variables and dynamically updates the similarity matrix, while the inner loop carries out manifold-regularized additive fitting. The approach substantially enhances robustness against noise and redundancy while improving predictive interpretability. Extensive experiments on four synthetic and twelve real-world datasets demonstrate its effectiveness, and theoretical analysis provides guarantees on algorithmic convergence and statistical generalization error bounds.

Technology Category

• Machine Learning: Semi-Supervised Learning
• Search and Optimization: Metareasoning and Metaheuristics
• Reasoning under Uncertainty: Graphical Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Semi-supervised learning with manifold regularization is a classical framework for jointly learning from both labeled and unlabeled data, where the key requirement is that the support of the unknown marginal distribution has the geometric structure of a Riemannian manifold. Typically, the Laplace-Beltrami operator-based manifold regularization can be approximated empirically by the Laplacian regularization associated with the entire training data and its corresponding graph Laplacian matrix. However, the graph Laplacian matrix depends heavily on the prespecified similarity metric and may lead to inappropriate penalties when dealing with redundant or noisy input variables. To address the above issues, this paper proposes a new \textit{Semi-Supervised Meta Additive Model (S$^2$MAM) based on a bilevel optimization scheme that automatically identifies informative variables, updates the similarity matrix, and simultaneously achieves interpretable predictions. Theoretical guarantees are provided for S$^2$MAM, including the computing convergence and the statistical generalization bound. Experimental assessments across 4 synthetic and 12 real-world datasets, with varying levels and categories of corruption, validate the robustness and interpretability of the proposed approach.

Problem

• semi-supervised learning
• manifold regularization
• graph Laplacian
• variable selection
• noisy input variables

Innovation

• Semi-supervised learning
• Manifold regularization
• Bilevel optimization
• Variable selection
• Graph Laplacian