MDBench: Benchmarking Data-Driven Methods for Model Discovery

2025-09-24

AI Summary

The lack of systematic benchmarks hinders rigorous evaluation of model discovery methods for dynamical systems. Method: This paper introduces the first open-source benchmark framework, encompassing 14 classes of partial differential equations (PDEs)—including seven high-challenge fluid dynamics and thermodynamics systems—and 63 classes of ordinary differential equations (ODEs), evaluated under multi-level noise with unified metrics. It integrates 12 state-of-the-art algorithms, including symbolic regression, sparse regression, and genetic programming, and quantifies performance via three orthogonal dimensions: derivative prediction accuracy, equation fidelity, and model complexity. Contribution/Results: Experiments reveal critical trade-offs: linear methods achieve the lowest error and highest robustness on PDE tasks, while genetic programming attains the best overall performance on ODE tasks. The benchmark fills a key gap in joint, noise-robust evaluation of PDEs and ODEs, establishing a standardized foundation to enhance reproducibility, comparability, and robustness of model discovery methodologies.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Machine Learning: Evaluation and Analysis
• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Model discovery aims to uncover governing differential equations of dynamical systems directly from experimental data. Benchmarking such methods is essential for tracking progress and understanding trade-offs in the field. While prior efforts have focused mostly on identifying single equations, typically framed as symbolic regression, there remains a lack of comprehensive benchmarks for discovering dynamical models. To address this, we introduce MDBench, an open-source benchmarking framework for evaluating model discovery methods on dynamical systems. MDBench assesses 12 algorithms on 14 partial differential equations (PDEs) and 63 ordinary differential equations (ODEs) under varying levels of noise. Evaluation metrics include derivative prediction accuracy, model complexity, and equation fidelity. We also introduce seven challenging PDE systems from fluid dynamics and thermodynamics, revealing key limitations in current methods. Our findings illustrate that linear methods and genetic programming methods achieve the lowest prediction error for PDEs and ODEs, respectively. Moreover, linear models are in general more robust against noise. MDBench accelerates the advancement of model discovery methods by offering a rigorous, extensible benchmarking framework and a rich, diverse collection of dynamical system datasets, enabling systematic evaluation, comparison, and improvement of equation accuracy and robustness.

Problem

• Benchmarking data-driven methods for discovering governing differential equations
• Evaluating model discovery algorithms on diverse dynamical systems with noise
• Addressing limitations in current methods for partial and ordinary differential equations

Innovation

• Introduced open-source benchmarking framework MDBench
• Evaluated 12 algorithms on 77 differential equations
• Assessed derivative prediction accuracy and noise robustness