Interactive Symbolic Regression through Offline Reinforcement Learning: A Co-Design Framework

2025-02-05

AI Summary

Symbolic regression (SR) suffers from combinatorial explosion, difficulty in incorporating domain expertise, and insufficient support for interactive, human-in-the-loop modeling. To address these challenges, we propose Sym-Q, a novel offline reinforcement learning framework for SR that departs from Transformer-based architectures and enables dynamic, user-guided editing of expression tree nodes at any stage—facilitating end-to-end human-AI co-design. Key contributions include: (i) a modular, plug-and-play tree encoder; (ii) an interaction-aware action space explicitly designed for co-design; and (iii) a physics-informed reward shaping mechanism that embeds domain constraints. On the SSDNC benchmark, Sym-Q substantially outperforms existing SR methods. In real-world applications, expert intervention further boosts accuracy, achieving superior performance over state-of-the-art models.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Neuro-Symbolic Learning
• Cognitive Modeling & Cognitive Systems: Symbolic Representations

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Symbolic Regression (SR) holds great potential for uncovering underlying mathematical and physical relationships from observed data. However, the vast combinatorial space of possible expressions poses significant challenges for both online search methods and pre-trained transformer models. Additionally, current state-of-the-art approaches typically do not consider the integration of domain experts' prior knowledge and do not support iterative interactions with the model during the equation discovery process. To address these challenges, we propose the Symbolic Q-network (Sym-Q), an advanced interactive framework for large-scale symbolic regression. Unlike previous large-scale transformer-based SR approaches, Sym-Q leverages reinforcement learning without relying on a transformer-based decoder. This formulation allows the agent to learn through offline reinforcement learning using any type of tree encoder, enabling more efficient training and inference. Furthermore, we propose a co-design mechanism, where the reinforcement learning-based Sym-Q facilitates effective interaction with domain experts at any stage of the equation discovery process. Users can dynamically modify generated nodes of the expression, collaborating with the agent to tailor the mathematical expression to best fit the problem and align with the assumed physical laws, particularly when there is prior partial knowledge of the expected behavior. Our experiments demonstrate that the pre-trained Sym-Q surpasses existing SR algorithms on the challenging SSDNC benchmark. Moreover, we experimentally show on real-world cases that its performance can be further enhanced by the interactive co-design mechanism, with Sym-Q achieving greater performance gains than other state-of-the-art models. Our reproducible code is available at https://github.com/EPFL-IMOS/Sym-Q.

Problem

• Addresses challenges in symbolic regression space
• Integrates domain experts' knowledge interactively
• Enhances performance with offline reinforcement learning

Innovation

• Reinforcement learning for symbolic regression
• Interactive co-design with experts
• Offline training with tree encoders