SyMANTIC: An Efficient Symbolic Regression Method for Interpretable and Parsimonious Model Discovery in Science and Beyond

2025-02-05

AI Summary

Symbolic regression (SR) faces critical challenges including high computational cost, poor scalability to high-dimensional problems, weak robustness to noise, and difficulty balancing accuracy against model simplicity. This paper proposes the first end-to-end framework integrating mutual information–based feature selection, adaptive feature expansion, and recursive ℓ₀ sparse regression. It introduces a Pareto frontier estimation method grounded in mutual information to explicitly model the trade-off between expression accuracy and structural complexity. Implemented in PyTorch with GPU acceleration, the framework is open-source and fully reproducible. Evaluated on synthetic datasets, scientific benchmarks, materials property prediction, and few-shot chaotic system identification, it achieves comparable or superior accuracy with significantly lower computational cost—reducing runtime by one to two orders of magnitude—while yielding compact, interpretable models. These results demonstrate its effectiveness and practical utility for scientific discovery.

Technology Category

• Machine Learning: Classification and Regression
• Search and Optimization: Mixed Discrete/Continuous Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Symbolic regression (SR) is an emerging branch of machine learning focused on discovering simple and interpretable mathematical expressions from data. Although a wide-variety of SR methods have been developed, they often face challenges such as high computational cost, poor scalability with respect to the number of input dimensions, fragility to noise, and an inability to balance accuracy and complexity. This work introduces SyMANTIC, a novel SR algorithm that addresses these challenges. SyMANTIC efficiently identifies (potentially several) low-dimensional descriptors from a large set of candidates (from $sim 10^5$ to $sim 10^{10}$ or more) through a unique combination of mutual information-based feature selection, adaptive feature expansion, and recursively applied $ell_0$-based sparse regression. In addition, it employs an information-theoretic measure to produce an approximate set of Pareto-optimal equations, each offering the best-found accuracy for a given complexity. Furthermore, our open-source implementation of SyMANTIC, built on the PyTorch ecosystem, facilitates easy installation and GPU acceleration. We demonstrate the effectiveness of SyMANTIC across a range of problems, including synthetic examples, scientific benchmarks, real-world material property predictions, and chaotic dynamical system identification from small datasets. Extensive comparisons show that SyMANTIC uncovers similar or more accurate models at a fraction of the cost of existing SR methods.

Problem

• Enhances symbolic regression efficiency and scalability
• Balances model accuracy with complexity effectively
• Handles large datasets and noise robustly

Innovation

• Mutual information-based feature selection
• Adaptive feature expansion strategy
• ℓ₀-based sparse regression application