LUMOS: Democratizing SciML Workflows with L0-Regularized Learning for Unified Feature and Parameter Adaptation

2026-02-25

AI Summary

This work addresses the inefficiency of scientific machine learning (SciML) modeling, which often relies heavily on extensive domain knowledge and manual hyperparameter tuning—particularly in feature selection and model sizing. To overcome this limitation, we propose LUMOS, a novel framework that introduces L0 regularization into an end-to-end joint optimization pipeline for the first time. By integrating a semi-stochastic gating mechanism with reparameterization techniques, LUMOS dynamically selects informative features and prunes redundant parameters during training, enabling a fully automated SciML workflow without manual intervention. Coupled with a distributed data-parallel strategy, LUMOS achieves an average parameter reduction of 71.45% across 13 SciML tasks, accelerates inference by 6.4×, and demonstrates strong scalability on an 8-GPU system.

Technology Category

• Machine Learning: Auto ML and Hyperparameter Tuning
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: ML for personalized search and recommendations
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

The rapid growth of scientific machine learning (SciML) has accelerated discovery across diverse domains, yet designing effective SciML models remains a challenging task. In practice, building such models often requires substantial prior knowledge and manual expertise, particularly in determining which input features to use and how large the model should be. We introduce LUMOS, an end-to-end framework based on L0-regularized learning that unifies feature selection and model pruning to democratize SciML model design. By employing semi-stochastic gating and reparameterization techniques, LUMOS dynamically selects informative features and prunes redundant parameters during training, reducing the reliance on manual tuning while maintaining predictive accuracy. We evaluate LUMOS across 13 diverse SciML workloads, including cosmology and molecular sciences, and demonstrate its effectiveness and generalizability. Experiments on 13 SciML models show that LUMOS achieves 71.45% parameter reduction and a 6.4x inference speedup on average. Furthermore, Distributed Data Parallel (DDP) training on up to eight GPUs confirms the scalability of

Problem

• Scientific Machine Learning
• Feature Selection
• Model Pruning
• Model Design
• Manual Tuning

Innovation

• L0-regularized learning
• feature selection
• model pruning
• scientific machine learning
• semi-stochastic gating