Adversarial LassoNet: Robust Feature Selection via Stability-Driven Sparse Learning

2026-07-04

AI Summary

Traditional ℓ₁-regularized feature selection suffers from instability under noise and spurious correlations, limiting its generalization capability. This work proposes a perturbation-driven stability optimization framework by deeply integrating adversarial training with LassoNet’s hierarchical sparsity mechanism. The approach enhances model robustness through first-order adversarial approximation and mitigates gradient concentration via spectral analysis inspired by the Neural Tangent Kernel (NTK). The resulting method substantially improves both the stability of feature selection and out-of-distribution (OOD) generalization. On ColoredMNIST, it achieves a 4.4% gain in OOD accuracy and a 6.3% improvement in feature support reproducibility; on a lung cancer screening dataset, it boosts test accuracy and AUC by 5.3% and 6.0%, respectively.

Abstract

Sparse feature selection is critical for high-dimensional machine learning, yet traditional $\ell_1$-regularized methods are often brittle under observational noise and spurious correlations, leading to unstable feature supports and degraded generalization. Although adversarial training has been widely used to improve model robustness, its interaction with hierarchical sparse feature selection remains underexplored. In this work, we propose Adversarial LassoNet (AdLNet), a stability-driven sparse feature selection framework that integrates input-space adversarial perturbations with the hierarchical sparsity mechanism of LassoNet. We derive a tractable first-order adversarial approximation under local smoothness assumptions and provide an NTK-inspired spectral analysis to characterize how perturbation-driven training can reduce gradient concentration. Experiments on high-dimensional SERS data, six public benchmark datasets, and ColoredMNIST show that AdLNet maintains competitive sparse-selection performance while improving out-of-distribution robustness by 4.4\% and feature support reproducibility by 6.3\% under nearly matched support sparsity on ColoredMNIST. On the high-dimensional lung cancer screening dataset, AdLNet achieves a 5.3\% test accuracy gain and a 6.0\% AUC improvement over vanilla LassoNet. Code and dataset are available at https://github.com/719573/Adversarial-LassoNet.

Problem

• sparse feature selection
• adversarial robustness
• high-dimensional data
• feature stability
• spurious correlations

Innovation

• Adversarial Training
• Sparse Feature Selection
• LassoNet
• Robustness
• Gradient Concentration