Call for Action: towards the next generation of symbolic regression benchmark

2025-05-06

AI Summary

Symbolic regression (SR) has long suffered from a lack of unified, dynamic, and reproducible benchmarks due to algorithmic heterogeneity, fragmented datasets, and inconsistent evaluation criteria. Method: We introduce SRBench—a next-generation open benchmark—systematically evaluating over 30 SR algorithms (nearly double prior benchmarks). It establishes the first tri-dimensional trade-off framework quantifying model complexity, accuracy, and energy consumption; standardizes hyperparameter optimization protocols, resource-constraint specifications, energy-monitoring toolchains, and execution sandboxes; and implements dynamic update mechanisms, algorithm deprecation policies, and community-driven curation workflows. Results: Empirical analysis reveals no universally dominant algorithm across all scenarios. SRBench currently encompasses 10+ datasets and 5+ evaluation dimensions, and has emerged as the de facto standard for the SR community—widely adopted and continuously evolving.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Evaluation and Analysis
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Symbolic Regression (SR) is a powerful technique for discovering interpretable mathematical expressions. However, benchmarking SR methods remains challenging due to the diversity of algorithms, datasets, and evaluation criteria. In this work, we present an updated version of SRBench. Our benchmark expands the previous one by nearly doubling the number of evaluated methods, refining evaluation metrics, and using improved visualizations of the results to understand the performances. Additionally, we analyze trade-offs between model complexity, accuracy, and energy consumption. Our results show that no single algorithm dominates across all datasets. We propose a call for action from SR community in maintaining and evolving SRBench as a living benchmark that reflects the state-of-the-art in symbolic regression, by standardizing hyperparameter tuning, execution constraints, and computational resource allocation. We also propose deprecation criteria to maintain the benchmark's relevance and discuss best practices for improving SR algorithms, such as adaptive hyperparameter tuning and energy-efficient implementations.

Problem

• Challenges in benchmarking Symbolic Regression methods due to diversity
• Need for standardized evaluation metrics and hyperparameter tuning
• Trade-offs between model complexity, accuracy, and energy consumption

Innovation

• Expands SRBench with more methods and metrics
• Analyzes model complexity, accuracy, energy trade-offs
• Proposes standardized hyperparameter and resource guidelines